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CHAPTER  I.  INTR{R>DCTION 


This  chapter  Introduces  the  report  with  a  statenent  of  the 
problem  and  discussion  of  the  need  followed  by  the  scope  of  the  research 
effort. 

A.  Statement  of  the  Problem 

The  problem  addressed  In  this  report  is  to  develop  and  demon¬ 
strate  a  methodology,  with  its  associated  computer  model,  that  will 
acceptably  transform  several  Individual  multicriteria  rank-ordered  lists 
of  research  and  development  (R  &  D)  projects  Into  a  single,  aggregated, 
prioritised  rank-ordered  list  to  guide  the  Investment  of  R  &  D 
resources.  In  addition,  provisions  are  needed  for  the  individual  lists 
to  be  converted  from  various  formats.  Decision-maker  and  judge  self¬ 
rating  weighting  methodologies  are  necessary.  The  methodology  developed 
most  be  capable  of  aggregating,  with  reasonably  small  effort,  very  long 
Individual  partial  length  and/or  full  length  lists.  Over  fifty  alter¬ 
natives  should  be  allowed  In  the  full  length  lists. 

B.  Need  for  this  Research  Solution 
The  task  of  R  &  D  management  planning  for  high  technology  systems 
has  become  exceedingly  difficult.  The  emphasis  on  coordination  and 
coMunlcatlon  between  the  management  groups  of  major  functional 
elements  of  high  technology  systems  developmental  organisations,  both 
Industrial  and  govensMntal,  and  the  expanded  usage  of  goal  and  objec¬ 
tive  planning  methods  have  eo^>llcated  the  planning  process.  Situations 
have  resulted  where  the  planners  In  the  R&D  element  receive  mousy  diverse 
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priority  llato  of  suggested  future  R  &  D  work  or  products  fro*  the 
other  fo&etlonsl  elenents  (l.e.,  aarfcetlngt  field  operations*  production* 
and  senior  staffs)  and  from  managers  within  the  R  &  D  organization. 

The  prioritization  criteria  of  interest  to  each  contributing  eleimnt  differ 
as  their  fuoctions  and  objectives  differ.  Therefore*  the  individual 
prioritizations  are  based  to  varying  extent*  upon  the  objective  criteria 
and  viewpoints  of  each  element.  The  R  &  D  element  management  must 
combine  these  lists  into  a  usable  list  of  prioritized  R  &  D  projects 
for  their  consideration  in  the  allocation  of  discretionary  R  &  D  funds 
for  use  toward  advancing  the  technology  base  of  the  enterprise. 

As  a  specific  example*  the  Director  of  the  US  Ar^  Missile  Labora¬ 
tory  has  such  a  planning  problem.  Each  year  he  selects  and  funds  up  to 
100  R  &  D  tasks  with  over  $50  million.  The  Army  policy  of  Single 
Program  Element  Funding  (SPEF)  gives  to  the  Laboratory  Director  the 
final  discretion  to  establish  the  most  productive  and  best  balanced 
program.  The  Director  is  provided  aa  abundance  of  advice  from  outside 
and  within  his  Laboratory.  The  advice  Is  usually  in  the  form  of  lists 
of  R  &  D  tasks  or  required  products  that  should  be  given  higher  priority 
and*  thereby*  funding.  To  advise  the  Army  Missile  Laboratory  Director* 
and  his  counterpart  at  other  Amy  R  &  D  Laboratories*  the  "user"  staff 
agencies  of  the  Amy*  which  are  the  Deputy  Chief  of  Staff  for  Operation 
(DCSOPS)  and  the  Training  and  Doctrine  Coowind  (TRADOC)*  annually 
prepare  docusMOtations  containing  their  priority  rankings  of  potential, 
not  yet  developed*  adlltary  materiel.  The  TRADOC  list  gives  a  raidc- 
order  number  to  each  future  system.  The  DCSOPS  document*  called  the 
Science  and  Technology  Guide  (STOG)  groups  the  potential  materiel 
systems  into  unranked  capability  category  classes  of  use;  l.e.*  Air 
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Defense*  Close  Coid>at*  Fire  Support*  etc.  Within  each  capability 
category*  the  potential  future  aysteos  are  rank  ordered.  The  Laboratory 
Directors  oust  relate  each  R  &  D  project  technology  task  to  the  one  or 
more  potential  future  systems  to  idilch  it  could  lead. 

Three  Headquarters  organizations*  the  Department  of  Defense  (DOD)* 
the  Department  of  the  Army  (DA)  *  and  the  Development  and  Readiness 
Command  (DARCOM)*  as  well  as  the  US  Congressional  staff  all  have*  on 
occasion*  sent  letters  to  Che  Director  reconmendlng  funding  of  certain 
selected  tasks  or  groups  of  tasks  within  the  Director's  SPEF  program. 

Trl- Service  special  topic  comnlttees  prepare*  and  often  prioritize* 
lists  of  R  &  D  project  tasks  which  they  recommend  for  Increased  funding. 
Axmy  Missile  R  &  D  technology  areas  of  emphasis  are  Included  In  the 
Trl-Servlce  committee  lists.  The  local  Commander's  staff  also  provides 
a  rank-ordered  list  of  systems  to  be  supported  by  the  Laboratory's  R  &  D 
technology  efforts.  Often  the  Comnander*  who  supervises  the  Laboratory 
Director*  has  a  few  R  &  D  technology  projects  or  potential  applications 
that  he  believes  should  be  given  special  attention  and  resources. 

Within  the  Army  Missile  Laboratory*  task  priority  rankings  are  prepared 
by  the  Director's  Staff  and  by  subordinate  managers  of  the  Directorates 
and  offices  within  the  Laboratory.  The  Laboratory  Director  must  allo¬ 
cate  his  discretionary  funds  to  the  R  &  D  technology  projects  that 
will  produce  the  most  return  for  Its  investment  cost  to  the  krmy  and 
will  maintain  the  viability  of  the  Laboratory.  He  must  give  appropriate 
managerial*  technical*  and  political  weight  to  each  of  the  recommenda¬ 
tions  of  his  advisors. 

The  preceding  statement  of  need  and  the  example  resolve  into  a 
desirability  for  a  methodology  for  aggregation  of  multiple  criteria  rank- 
ordered  priorities. 
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C.  Scope  of  this  Research 

The  research  documented  by  this  report  Is  based  upon  a 
comprehensive  literature  survey  of  the  subjects  of  social  choice  and 
majority-rule  methods  that  are  applicable  to  the  aggregation f  without 
feedbacks  of  multiple  criteria  rank-ordered  ordinal  priorities.  From 
this  basis,  the  research  will  determine  and  develop  the  specific 
majority-rule  methodology  to  aggregate  the  variety  of  rank-ordered 
priority  lists  (as  described  previously)  for  R  &  D  project  priority 
determination.  The  chosen  majority-rule  methodology  will  be  Integrated 
Into  an  aggregation  logic  model  that  will  satisfy  the  following  require¬ 
ments: 

1)  Aggregate  rank-ordered  Individual  subllsts  which  have  any  or 
all  of  the  following  features: 

a)  Complete  length  lists  of  up  to  100  alternatives  that 
rank  all  possible  projects  or  requirements  for  products. 

b)  Reduced  length  lists  (down  to  two  alternatives)  that 
rank  less  than  the  complete  set  of  possible  projects  or  requirements. 

c)  Transitive  rank-ordered  subllsts. 

d)  Weak  ordered  and/or  strongly  ordered  (X  ^  Y  and/or  X  >  Y) 

subllsts . 

e)  Categorized  grouping  subllsts,  where  one  of  the  following 

may  occur: 

(1)  The  projects  are  subdivided  Into  categories.  Then, 
within  each  category,  the  projects  are  ranked,  but  the  categories  may 
or  may  not  be  strongly  ranked. 

(2)  The  projects  are  subdivided  Into  categories.  The 
categories  are  ranked,  but  the  projects  within  a  category  nay  or  nay 
not  be  strongly  ranked. 
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£)  Multlpls  subllsts  ranked  in  accordance  with  a  coi— on 
criteria  or  with  individual  criteria. 

g)  Subllsta  where  the  altemativea  are  ordinal-ranked  baaed 
upon  various  foias  of  cardinal  utiUtlea  auch  as: 

(1)  Value  estimates,  or, 

(2)  The  date  that  the  usable  materiel  product* from  the 
R  &  D  will  be  available. 

2)  Weight  the  importance  and  authenticity  of  each  sublist  during 
the  aggregation  process.  The  types  of  weighting  mechanisms  Include 
the  following: 

a)  Declslott-maker  weighting  mechanisms  to  be  applied  to 
single  alternative  and/or  to  the  subllsts  ranked  by  certain  judges. 

The  mechanisms  will  Include  multiplicative  factors  and  exponential  factors. 

b)  Judge  self-expertise  weighting  where  each  judge  %d.ll  rate 
his  own  expertise  on  each  alternative. 

3)  Analytically  measure  and  statistically  test  the  concordance 
of  the  set  of  subllst'rank  orders  and  the  consistency  of  the  aggregate 
rank  order. 

The  logic  model  was  converted  Into  a  digital  computer  code  to 
perform  the  preceding  requirements  for  up  to  100  rank-ordered  subllsts. 

The  final  logic  model  and  computer  code  will  be  demonstrated  In  an 
R  &  D  project  prioritization  study  (see  Chapter  VII)  tailored  to  resolve 
the  Army  Missile  Laboratory  Director's  concerns  as  described  In  Section  B 
of  this  chapter. 


CHAPTER  II.  SURVEY  OF  PAST  RELATED  LITERATURE 

This  chapter  contains  the  results  of  a  comprehensive  survey  of 
the  literature  related  to  the  research  topic.  The  chapter  contains  a 
chronological  and  network  analysis  of  the  literature  thrusts  as  well 
as  synopses  of  key  material.  An  extensive  annotated  bibliography  was 
prepared  and  published  In  another  report,  Dobbins  [13].* 

A.  Overview  of  the  Literature 

Extensive  available  literature  was  surveyed  to  select  and  evaluate 
the  "Social  Choice"  relevant  body  of  knowledge  and  Its  application  to 
the  problem  of  the  aggregation  of  several  ordinal  R  &  D  project  prefer¬ 
ence  rank  orders  Into  a  single  rank-ordered  list.  This  final  rank 
order  should  provide  a  reasonable  representation  of  the  consensus  of 
the  Individual  preferences.  Since  the  known  literature  was  primarily 
in  the  fields  of  welfare  economics,  political  science,  and  social  science. 
It  had  to  be  Interpreted  and  translated  to  determine  Its  relevance  to 
R  &  D  engineering  management.  For  a  better  understanding  of  the 
material  and  for  the  application  of  the  available  knowledge,  the  relevant 
literature  was  separated  Into  distinct  thrust  areas  of  emphasis.  These 
thrtist  areas  then  were  Interrelated  on  a  time  axis  to  determine  when 
and  where  the  later  thrusts  branched  off  from  the  initial  endeavors. 


*[  ]  denotes  the  nunber  of  a  book  or  periodical  cited  in  the 
references. 
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The  search  of  the  extensive  relevant  literature  did  not  identify 


any  work  dealing  directly  with  the  goal  of  this  report.  There 

were  no  majority-rule  type  methods  for  aggregation  of  multiple  criteria, 

ordinal  rank-ordered  lists  to  obtain  a  single  rank-order  list  of  R  &  D 

projects  for  resource  investment.  A  few  articles  did  recognize  that 

R  &  D  project  selection  was  a  possible  application  for  social  choice 

theory. 

Certain  restrictions  were  used  to  select  the  relevant  material 
from  the  extensive  body  of  social  choice  information.  These  restrictions 
Include  the  following: 

1)  The  rank  orders  of  Interest  were  ordinal,  not  cardinal, 
preferences. 

2)  The  emphasis  was  on  the  aggregation  of  Individual  ranked 
preferences  Into  single  group  rank-order  lists. 

3)  The  rank  ordering  and  aggregation  were  single  cycle,  not 
temporal,  decisions. 

4)  There  was  no  feedback  from  Individuals  to  other  group  members. 

5)  The  individuals  who  ranked  applied  their  sincere  beliefs  and 
were  not  using  strategy  to  coerce  the  group  result  to  agree  with  their 
preferences  or  objectives. 

The  literature  selected  was  grouped  Into  25  thrust  area  groups  of 
Interest  and  of  manageable  size.  The  reference  material  was  chrono¬ 
logically  sequenced  within  each  thrust  area.  To  obtain  an  Indication 
of  the  time  span  of  work  In  each  area,  the  dates  of  the  earliest,  the 
median,  and  the  latest  paper  in  each  group  were  recorded  In  Table  1. 

Also  listed  Is  the  number  of  papers  and  books  relevant  to  this 
report  that  were  grouped  In  each  area.  The  first  area,  a)  Pre-Arrow 
Basic,  had  four  works  that  span  from  1953  translations  of  the  1770 
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papers  to  1958  with  a  nedlan  paper  dated  1953.  The  last  area,  y)  Ihssy 
Set  Raok  Ordering,  had  ten  papers  irtilch  span  fron  1974  to  1979  with  a 
1978  median  data.  Except  for  two  areas,  Pre-Arrow  Basle  and  Tullock's 
Books,  all  of  the  work  areas  appeared  to  be  active  at  the  writing  of 
this  report  and  more  material  can  be  reasonably  expected  to  be 
published. 

Figure  1  presents  a  network  visualization  of  the  Interrelation 
along  a  time  axis,  of  the  25  literature  thrust  areas.  More  specialized 
areas  branch  off  from  d)  Majority-Rule  Models  thrust.  The  o)  Thllock's 
Book's  activity  starts  and  ends  within  the  time  scale  of  this  network. 
The  network  began  to  branch  rapidly  during  the  period  insiedlately  after 
the  publications  of  Arrow's  Impossibility  Theorem.  The  current  continu¬ 
ation  of  work  In  23  of  the  25  areas  and  the  relatively  late  median  dates 
listed  In  Table  1,  reflect  the  accelerating  activities  of  the  1970 's, 
possibly  kindled  by  the  several  excellent  bookS'  published  since  the 
mld-1960's  by  authors  such  as  Sen  and  Flshburn. 

B.  Synopses  of  Literature 

The  following  paragraphs  will  give  a  brief  synopsis  of  over  200 
books  and  articles  selected  as  relevant  to  the  research  for  this 
report.  Each  book  and  article  Is  summarized  In  somewhat  more 
detail  In  Dobbins  [13].  All  are  listed  In  the  References.  The 
material  presented  In  each  area  of  the  synopses  and  Dobbins  [13]  has 
been  discussed  chronologically  according  to  the  publication  year.  The 
synopses  are  as  follows: 

1)  Pre-Arrow  Basic:  Methods  to  use  the  preferences  of  the 
majority  of  voters  to  obtain  a  single  consensus  apparently  were  recorded 
[101]  first  In  1770  when  Borda  recommended  his  "method  of  marks." 
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Condorcet's  true  majority  and  LaPlace's  method  followed  closely  after 
Borda's  work.  Dodgson  (Lewis  Carroll)  recomnended  modifications  to 
Borda's  method  during  the  mid-1800's.  The  economic  interest  in 
rank-ordered  aggregation  was  kindled  through  Robbins'  [31]  1932 
contention,  supported  further  by  Bergson  [58] ,  that  the  magnitudes  of 
individual  preferences  cannot  be  added  but  must  be  analyzed  by  ordinal 
means . 

2)  Statistical  Correlation  Methods:  Kendall,  Smith,  FriedsMn, 
and  Moran  [22,  132,  161,  162,  163,  183]  developed  the  statistical  rank- 
ordered  correlations,  such  as  the  coefficient  of  concordance  method. 

This  work  has  been  further  covered  by  others  [15,  16,  19,  24,  26,  106]. 

3)  Single-Peakedness:  Black's  contention  [4,  62,  63]  that 
single-peaked  orders  had  preferred  aggregation  characteristics  was 
substantiated  and  expanded  upon  by  others  [6,  23,  74,  103,  104]. 

4)  Majority-Rule  Methods:  Various  majority  methods  have  been 
developed  either  as  derivatives  of  the  classical  methods  (Borda, 
Condorcet,  etc.)  or  as  new  approaches.  These  include  a  vote  score 
assignment  equation  by  Schuler  [210],  dichotomous  voting  by  Bartoszynski 
[56],  pattern  classification  byBlinand  Whlnston  [73],  an  aspiration 
level  overlay  by  Harnett  [146],  a  majority-rule  Kendall-derived  two- 
matrix  method  by  Shannon  [217],  an  extension  beyond  Shannon  by 
Havarrete,  et  al.  [185],  a  point  system  by  Smith  [221],  a  branch  and 
bound  algorithm  method  by  Armstrong  and  Cook  [50],  others  [88,  95,  126, 
143,  220,  225,  227],  and  a  conversion  from  ordinal  to  cardinal  rank 
orders,  before  aggregation,  method  by  Wood  and  Wilson  [237]. 

5)  Number  of  Voters  and  Alternatives:  Flshbum  [120,  124,  131] 
and  Bell  [57]  developed  several  papers  that  quantify  the  likelihood 
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that  a  staple  majority  method  will  produce  a  winner  as  the  quantity  of 


voters  and  alternatives  vary.  Black,  [64],  Dutta  and  Pattanlak  [105], 
and  Greenberg  {141]  added  theoretical  depth  to  these  data. 

6)  Arrow's  laiposslblllty  Theorem:  K.  Arrow,  in  1951,  determined 
[1,  51]  the  required  conditions  for  a  social  welfare  function  which 
aggregated  Individual  preferences,  then  proved  the  theorem  that  a  fair 
social  welfare  function,  without  a  dictatorship,  was  impossible.  The 
required  conditions  were  more  simply  and  clearly  stated  by  Little  [169] 
as  follows: 

a)  Retrlevablllty  of  alternatives. 

b)  An  alternative's  relative  position  In  sn  Individual 
order  will  not  relatively  change  in  the  aggregate  order. 

c)  The  Independence  of  Irrelevant  alternatives. 

d)  Non-imposition. 

e)  Non-dic  ta tor ship . 

Numerous  researchers  countered  his  theorem  [82,  189] ,  offered  modifi¬ 
cations  (99,  108,  181],  or  presented  alternative  proofs  [111,  173, 

179,  211]  of  Arrow's  results. 

7)  Estenslons  and  Revisions  of  Arrow's  Impossibility  Theorem: 
While  striving  to  overcome  the  "impossible,"  social  science  researchers 
developed  new  or  modified  conditions  for  social  welfare  functions  that 
could  be  satisfied,  albeit  from  a  reduced  variety  of  acceptable  indivi¬ 
dual  orders.  May  [171]  defined  conditions  of  decisiveness,  symmetry, 
neutrality,  and  positive  responsiveness.  Other  conditions  defined  are 
simsHitlon  of  raidis  [94],  unanimity  and  monotonlcity  [67],  split  groups 
of  Indifferent  alternatives  [152],  Intensity  added  to  the  Independence 
of  Irrelevant  alternatives  [86],  Intensity  of  antagonism  index  [168], 
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anonymity  [110],  group  rationality  [109].  Others  reaffirmed  Arrow's 
laqposslblllty  results  through  different  assui^tions  [53,  66,  75,  97, 

115,  144,  145,  148,  151,  191,  213,  214,  233,  234,  235]. 

8)  General  Social  Choice  Theory  Overview:  As  knowledge  developed, 
several  comprehensive  social  choice  theory  works  became  available  from 
authors  such  as  Luce  and  Ralffa  [27],  Pattanalk  [28,  29],  Flshbum  [17], 

.  Blln  [69],  Herzburger  [147],  Ferejohn  and  Page  [112],  and  Aumaim  [3]. 

9)  Social  Welfare  Function:  The  definitions  and  conditions  for 
social  welfare  functions  have  been  analyzed  [2,  102,  114,  116,  135, 

160,  193,  239],  while  the  question  of  the  existence  of  such  functions 
has  been  debated  [158,  205,  208]. 

10)  Comparison  of  Borda  and  Condorcet:  In  recent  years  zuiny 
researchers  have  comparatively  analyzed  the  majority-rule  methods 
developed  by  Borda  and  Condorcet.  De  Grazla  [101]  and  Black  [4] 
provided  translations  and  analyses  of  the  original  papers.  Flshbum 
[119,  129],  Young  [238,  240],  and  Richelson  [200,  202,  203]  compared 
the  two  methods.  The  Condorcet  method  was  the  reference  method  of 
several  papers  [129,  137,  190],  while  Borda's  method  was  used  for 
others  [49,  77,  94,  116,  121,  125,  130,  180,  241]. 

•  11)  Majority  Rule-Multiple  Methods  Compared:  Goodman  described 

[139]  the  Copeland  two-step  method  and  compared  It  favorably  to  other 
methods,  as  did  Richelson  [200,  202,  203].  Svestka  [40],  Ityatt  [47], 
Chartler  and  Wertheimer  [91],  and  Castore,  Peterson,  and  Goodrich  [90] 
made  comprehensive  comparisons  of  lists  of  majority  rule  and  other 
aggregation  methods. 

12)  Transitivity,  Intransltlvlty,  and  Cyclicity:  An  accepted 
limitation  of  the  unrestricted  use  of  many  majority-rule  methods  Is 
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the  real  probability  of  aggregated  results  which  are  intransitive  or 
cyclic.  Many  researchers  have  worked  to  better  understand  the  charac¬ 
teristics  and  causes  of  transitivity »  intransltlvlty*  and  cyclicity, 
and  to  develop  conditional  restrictions  that  will  control  these  aspects 
of  rank-ordered  aggregation.  May  [172]  and  others  strived  to  clarify 
Intransltlvlty  [117,  156,  174,  212],  while  Inada  and  others  [76,  98,  100, 
118,  140,  153,  155,  157,  167,  178,  198,  209,  215]  developed  decision 
rules  to  avoid  Intransitive  results,  generally  through  restrictions  on 
the  characteristics  of  the  individual  rank  orders.  Pomeranz  and  Well 
[196]  computed  probabilities  of  cyclical  majorities. 

13)  Basic  Arrow  and  Majority  Rule  Theory:  New  axiomatic  structures 
for  preference  theory  were  developed  by  Luce  [170],  Fishbum  [114,  122, 

123,  127,  128,  140],  and  others  [93],  while  Koopman  [166]  clarified 
concepts  in  existing  theory. 

14)  Majority  Rule-Minimum  Loss  Methods:  Van  den  Bogaard  and 
Versluls  [44]  and  others  [21,  150,  227]  developed  techniques  to  aggregate 
individual  welfare  preferences  by  minimization  of  the  social  loss  function 
which  was  based  upon  Individual  loss  functions. 

15)  Survey  Literature:  Surveys  of  the  work  and  publishings  of 
others  in  the  field  were  made  by  Rlker  [in  Coombs  10,  199],  Gullbaud 

in  Lazerfield  and  Henry  [25],  Sen  [34],  Flshburn  [17],  Plott  [195],  and 
others  [6,  7,  8,  11,  30,  32,  33,  39,  42,  81]. 

16)  Majority  Rule-Game  Theory  and  Strategy  Proof ness:  Based 
upon  the  1947  work  of  von  Neumann  and  Morgenstem  [45],  game  theory 
had  been  applied  to  the  simple  isajorlty  concept.  Barbut  [54]  intro¬ 
duced  the  relation  of  the  two-person,  zero-sum  game  to  majority-rule 
methods,  while  Shisko  [36]  extended  this  approach  for  n-person  majority-rule 
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gaaes.  The  relevant  theory  was  developed  and  extended  [84,  85,  175,  176, 
184,  206,  207].  Glbbard  [136]  applied  the  game  theory  approach  to  develop 
the  criteria  for  a  strategy-proof  voting  scheme. 

17)  Tullock's  Books:  G.  Tullock  stated  In  his  books  [5,  43,  228] 
that  Arrow's  Impossibility  Theorem  would  seldom  be  Important  and  that 
little  had  been  contributed  In  the  majority-rule  area  since  Black 
developed  slngle-peakedness .  Tullock's  critical  writings  generated 
responses  from  Arrow  [52]  and  others  [20,  55]. 

18)  Majority  Rule  Examples:  A  few  reports  were  found  that  repre¬ 
sented  practical  examples  of  the  application  of  the  majority  rule  [87, 

213,  229]. 

19)  Voter's  Paradox:  Effort  has  continued  to  generalise  the 
analysis  to  determine  the  probability  of  achieving  an  intransitive 
aggregate  rank-order  through  a  majority-rule  method  [48,  89,  133,  134, 

164,  186,  231]. 

20)  Majority-Rule  Conditions  and  Equilibrium-Diverse  efforts  have 
continued  striving  to  determine  the  coiklltlons  required  for  a  usable 
majority-rule  decision.  Murakami  [182]  determined  that  the  scoring 
constants  should  logically  be  1,  0,  -1,  have  autonomy,  be  non-reversed, 
and  have  non-dictatorship.  Pattanalk  [192]  pursued  the  value  restrictions 
techniques,  while  Fine  [113]  recommended  monotonlclty  and  faithfulness. 
Blau  and  Deb  [68]  proved  that  an  Infinite  social  decision  function  does 
not  produce  a  choice.  Rlchelson  [201]  classified  and  showed  the  inter¬ 
relations  of  the  multitude  of  overlapping  conditions  for  a  social  choice 
function.  Others  [35,  38,  149,  159,  177,  187,  197,  216,  219]  also 
worked  In  this  area. 
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21)  Weighting  Methods:  Weighting  permits  special  lo^ortance  to 
be  considered  for  certain  Judges'  rankings  or  for  certain  alternatives. 
Winkler  [236]  and  others  [92,  165]  summarized  the  most  useful  methods: 
equal  weights,  weights  proportional  to  ranking,  weights  proportional 

to  self -rating,  and  weights  based  on  previous  assessments.  Gustafson, 
Pal,  and  Kramer  [142]  reported  on  a  weighting  method  for  hlerarachical 
R  &  D  project  selection.  Rowse,  Gustafson,  and  Ludke  [204]  added  peer 
weights,  group  weights,  and  average  weights  to  the  list  of  available 
methods.  Elnhorn  and  Hogarth  [107]  discussed  unit  weighting. 

22)  Majority  Rule-Graphical  Methods:  Research  has  continued 
toward  describing  the  social  decision  process  through  graphical 
structures  [21,  46,  138,  224,  232]. 

23)  Resource  Allocation  by  Voting:  Several  studies  were  made 
of  the  effect  of  voting  rules  on  resource  allocation  processes  such 
as  capital  budgeting  and  R  &  D  project  selection  [12,  18,  61,  80,  194, 
222]. 

24)  Majority  Rule-Minimum  Distance  Technique:  Bowman  and 
Colantonl  [78,  79]  developed  a  majority-rule  rank-ordered  decision 
method  based  upon  the  minimizing  of  a  decision  function  defined  as  a 
one-dimensional  "distance."  The  scoring  constants  used  are  1,  1/2,  0 
and  the  distance  function  Is  d  (p*,  P)  ■  f 

and  Whlnston  [72]  suggest  a  quadratic  assignment  solution  to  the  Bowman 
and  Colantonl  method.  Cook  and  Selford  [96]  extended  the  one-dlmenslonal 
distance  minimization  method  so  that  It  can  be  solved  by  linear  program¬ 
ming  techniques. 

25)  nizzy  Set  Rank  Ordering:  Blln  originally  proposed  [70,  71] 
that  fuzzy  binary  relations  could  be  used  for  group  preference  orderings. 
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Others  [154,  188,  223,  230,  233]  especially  Bezdek,  Spillman,  and 
Spillman  [59,  60]  have  developed  Blln's  proposition  into  a  new  area  of 
social  choice  study.  Buckles  [83],  as  class  work,  translated  from 
Fuzzy  Set  terminology  to  that  of  majority  rule. 
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CHAPTER  III.  PRIORITIZED  RANK  ORDER  AGGREGATION 
METHOD  CONCEPT 

In  this  chapter,  the  concepts  of  ordinal  aggregation  are  presented 
by  first  Introducing  the  foundation  considerations,  then  presenting  a 
comparative  analysis  of  majority-rule  methods. 

A.  Foundation  Considerations 

1.  Need 

The  majority-rule  concept  chosen  for  the  objective  method 
of  this  research  must  be  flexible  In  Its  application  as  well  as  techni¬ 
cally  satisfactory.  It  must  accept  complete  and  partial  orders,  strong 
and  weak  preferences,  and  weighting  factors.  Also,  It  must  not  be 
negated  by  Intransltlvltles.  The  quality  of  the  whole  aggregated  rank 
order  Is  more  significant  than  the  precision  of  the  first  choice 
winner  selection.  For  resource  allocation  purposes  where,  for  example, 
the  first  40  projects  are  ranked,  the  Identity  of  the  number  one  project 
Is  of  passing  Interest,  while  being  In  the  first  segment  (say,  the 
first  lOZ)  can  be  of  pressing  Importance. 

2.  Known  Method  Characteristics 

Various  aggregation  rank-ordered  generating  methods  relevant 
to  this  research  have  been  surveyed  and  seven  have  been  chosen  for 
comparative  analysis.  None  of  the  methods  chosen  utilised  feedback 
of  grotip  consensus  to  the  judges  nor  dialogue  between  Judges.  Further, 
none  of  the  chosen  methods  required  cardinal  utilities  to  measure 
preferences.  These  restrictions  also  eliminated  the  Delphi  method  and 
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the  unfolding  method  offered  by  Coombs  [10].  The  family  of  methods 
considered  further  was  variations  of  the  majority-rule  method.  For 
problems  with  only  two  alternatives  these  laethods  all  resolved  to 
the  same  simple  majority-rule  method  where  the  winner  was  the  alterna¬ 
tive  that  had  the  majority  of  the  votes.  But  for  three  or  more 
alternatives,  several  defined  methods  were  available.  These  majority- 
rule  methods  are  as  follows: 

a)  Borda's  Method  of  Marks. 

b)  Adjusted  Borda  Method  of  Marks. 

c)  Condorcet's  Criterion. 

d)  Black's  Simple  Majority  Procedure. 

e)  Dodgson's  Method  of  Inversion. 

f)  Copeland's  Majority  Buie. 

g)  Shannon  and  Svestka's  Majority  Buie  Method. 

h)  Black's  Single-Peaked  Preference. 

Further  examination  of  the  theory  and  practice  of  h)  Black's 
Single-Peaked  Preference  determined  that  It  was  not  a  practical  method 
for  aggregation  with  a  large  number  of  alternatives  because  the  only 
known  method  required  consideration  of  all  alternative  combinations 
as  possible  reference  orders.  After  an  explanatory  description  in 
Section  4  of  this  chapter,  this  mthod  will  not  be  evaluated. 

Majority-rule  methods  were  recommended  by  Kenneth  Arrow 
as  the  better  method  for  social  choice  aggregation.  Arrow's  bo<dc 
[1]  is  credited  by  Blau  [67]  with  presenting  the  first  organisation  of 
the  social  choice  problem  as  a  deductive  system.  Since  interpersonal 
comparison  of  utility  had  been  excluded,  the  theory  of  ordinal  relations 
became  dominant  for  social  ordering.  Arrow's  work  showed  that  when  the 
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axioas  (conditions)  of  econoalc  social  choice  were  fully  defined,  they 
were  Inconsistent.  Arrow  defined  the  Social  Welfare  Function  (SWF) 
as  a  technique  to  express  the  aggregate  econowlc  preferences  of  society. 
Arrow's  conditions  for  SUF  were  the  foundation  of  the  current  social 
welfare  theory,  but  they  were  considered  by  aany  writers  as  too  restric¬ 
tive.  His  conditions  were  highly  technical  and  difficult  to  understand. 
Therefore,  each  Arrow  condition  will  be  replaced  by  a  wore  easily 
understood  Interpretation  by  Svestka  [40] . 

Condition  1: 

A  social  welfare  function  (aaalgaiBatlon  method)  oust  define 
a  unique  order,  given  any  subset  of  all  possible  orders. 

Condition  2: 

If  one  alternative  rises  or  remains  stationary  In  the  order 
of  every  Individual,  then  It  must  iu>t  fall  In  the  joint  order. 

Condition  3: 

The  removal  or  Insertion  of  an  alternative  In  the  set  of 
alternatives  which  result  in  no  change  in  any  Individual  order  of  the 
remaining  alternatives  must  not  cause  a  change  In  the  order  of  the 
remaining  alternatives  of  the  Joint  order. 

Condition  4: 

The  Joint  order  is  a  function' of  the  individual  orders  and 
oust  not  be  imposed  by  some  outside  Influence. 

Condition  5: 

The  joint  order  must  not  be  arbitrarily  defined  by  the  order 
of  one  individual  without  consideration  of  the  other  Individual  orders. 

Many  theoretical  writers  responded  to  Arrow's  c<mdltlons  published 
In  1951.  Based  on  certain  of  these  critiques.  Arrow  revised  three  of 
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his  conditions.  Conditions  1,  2,  and  5  were  strengthened,  but  the 
condition  inconsistency  represented  by  Arrow's  Impossibility  Theorem 
still  held.  Arrow  further  wrote  that  for  more  than  two  alternatives 
the  method  of  majority  (majority  rule)  satisfied  Conditions  2  through  5 
and  a  restricted  form  of  Condition  1  (labeled  1'),  and  thus  was  like  a 
social  decision  function,  even  if  not  an  SWF.  Arrow  did  not  specify 
precisely  a  majority-rule  method  for  more  than  two  alternatives. 
Promising  methods  from  the  literature  will  be  described  later  in  this 
chapter.  To  aid  in  understanding  the  material  to  follow,  certain 
social  choice  notation  will  be  explained: 

xPy  -  means  that  the  alternative  x  is  strictly  preferred  to  the 
alternative  y  (x  >  y) . 

xlty  -  means  that  in  paired  comparative  voting,  alternative  x  has 
a  majority  of  the  votes  over  alternative  y. 

xRy  -  means  that  alternative  x  is  equal  to  or  preferred  to  alter¬ 
native  y  (x  >  y) . 

xly  -  means  that  the  judge  is  indifferent  between  alternative 
X  and  alternative  y  (x  ■  y) . 

3.  Comparison  Characteristics 

The  seven  majority-rule  methods  will  be  compared  to  determine 
the  one  that  is  preferred  for  the  aggregation  model  for  this  research. 
They  will  be  compared,  similar  to  Vtyatt  [47]  and  Rlchelson  [200,  202, 
203] ,  by  the  requirements  that  must  be  met  by  the  individual  sublist 
orders  before  the  aggregation  method  can  be  utilized,  and  by  conditions 
of  majority-rule  performance.  The  requirements  and  procedural  condi¬ 
tions  considered  for  the  methods  are  as  follows: 
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a)  Only  Cardinal  Utllltlea  Required?  -  Must  the  aethod 
accept  only  cardinal  rank  order  data,  or  are  ordinal  data  accepted? 

b)  Only  Transitive  Orders  Required?  -  Can  the  aethod  aleo^ 
accept  Intranaltlve  rank  orders? 

c)  Only  Coiiq>lete  Orders  Required?  -  Can  the  aethod  also 
accept  partial  orders? 

d)  Only  Strong  Orders  Required?  *•  Can  the  aethod  also  accept 
weak  rank  orders? 

e)  Restricted  Number  of  Judges  or  Alternatives?  -  Is  the 
aethod  restricted  to  an  odd  or  even  nuaber  of  Judges  or  alternatives, 
for  escaaple? 

f)  Not  Have  Condorcet  Extension  Procedure?  -  If  an  altexna-* 
tlve  has  a  simple  majority  over  every  other  alternative,  will  it  be 
the  first  place  choice?  (See  Richelson  [200]). 

g)  Not  Have  Cancellation  Property?  -  Will  any  yPx  in  an 
individual's  preference  ordering  be  balanced  by  any  xPy  in  soae  other 
individual's  ordering  as  long  as  there  are  no  other  alternatives 
between  x  and  y  in  either  ordering?  (See  Richelson  [200]). 

h)  Difficult  to  Enlarge?  •  Is  the  implementation  of  the 
method  significantly  more  difficult  as  the  number  of  Judges  or  number 
of  alternatives  Increases  from  three? 

1)  Produces  Only  Winners?  -  Is  the  method  Intended  to 
produce  full  aggregated  rank  orders  or  only  the  first  place  alternative? 

4.  Descriptions  of  Compared  Methods 

The  following  is  a  more  detailed  discussion  of  the  majority- 
rule  methods  coaq>ared  in  this  dissertation. 
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a 


Borda's  Mechod  of  Marks 


Black  [4,  65]  Interpreted  Jean-Char les  de  Borda's  paper, 
printed  in  1781  In  the  Memoirs  of  the  French  Academy  of  Science,  to 
offer  two  methods  which  gave  the  sasm  results.  The  second  of  Borda's 
stated  methods  gave  each  alternative  a  mark  equal  to  the  sum  of  the 
votes  that  It  would  get  when  It  was  put  against  each  of  the  other 
alternatives.  Individually.  When  the  alternatives  were  z,  y,  and  s. 

If  y  were  placed  first  In  the  preferences  of  any  given  judge.  It  would 
defeat  x  and  z  and  would  secure  two  votes.  If  x  stood  second  In  the 
judge's  preference.  It  would  defeat  one  alternative  and  secure  one  vote. 
This  method  had  been  demonstrated  by  Black  to  hold  for  strong  orders. 

Flshbum  [119]  and  Rlchelson  [200]  both  defined  Borda's  social 
decision  function  as: 


f(Y,D)  ■  |x:xeY  and  Z  r. (x,Y,D)  >  Z  r. (y,Y,D)  for  all  yeYj 
'  1-1  “  1-1  ' 

where  r^  Is  the  rank  assigned  to  each  alternative  by  judge  1,  Y  Is  the 

set  of  feasible  alternatives,  and  D  Is  the  n-tuple  of  Individual  linear 

orderings  on  Y.  For  each  alternative,  the  Borda  score  was  the  sum  of 

the  marks  It  received  from  the  judges,  where,  for  m  alternatives,  the 

marks  given  were.  In  decreasing  order  of  preference,  (m-l>,  (m-2)  ...,  0. 

The  ranking  was  based  on  the  scores  of  each  alternative;  l.e.,  the 

highest  score  won. 

Example  1: 

An  example  of  the  Borda  method,  with  N  -  5  judges  and  M  -  4 
alternatives  Is: 

Judge  1:  w>x>y>s 
Judge  2:  x>y>z>w 
Judge  3:  w>x>y>z 
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Judge  4:  v  >  x  >  z  >  y 
Judge  5:  x>s>y>w 

By  competing  each  alternative  againat  all  other  alternatlvea*  the 
following  matrix  of  votes  Is  obtained: 

Borda 
Row 

w  X  y  z  Totals 
w  p  3  3  3 

X  2  -  5  5 

y  2  0  -  3 

z  I  2  0  2  - 

In  this  matrix,  for  example,  when  x  competed  against  w,  there 
were  3  votes  for  w  over  x  and  2  votes  for  x  over  w.  The  Borda  winner 
for  the  example  problem  Is  x  with  12  votes  total  and  w  Is  second  with  9. 
The  Borda  rank  order  Is  x  >  w  >  y  >  z. 

b.  Adjusted  Borda*s  Method  of  Marks 

Black  [4,  65]  further  Interpreted  Borda 's  work  to  cover 
orders  where  soow  Judges  were  Indifferent  between  certain  alternatives. 
Black  explained  the  Adjusted  Borda  method  as  that  of  assigning  marks 
on  the  basis  of  one  mark  for  each  alternative  It  stood  above  and 
deducting  one  mark  for  each  alternative  It  stood  below.  In  terms  of 
the  matrix  analysis.  Black  said  that  this  method  assigned  to  an  alter¬ 
nate  a  plus  mark  equal  to  the  total  nuaber  of  alternatives  above  its 
place  on  all  of  the  sublists,  and  a  minus  mark  equal  to  the  total  of 
the  alternatives  below  its  place.  The  plus  portion  could  be  calculated 
by  summing  the  figures  in  the  row  cells  for  that  alternative  and  the 
minus  portion  was  obtained  from  summing  the  column  calls  for  that 
alternative . 


9 

12 

5 

4 
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Flshburn  [119]  defined  the  Adjusted  Borda  social  decision  function 

as 

n  n 

P(Y,D)  -  (x:xeY  and  E  S(x,Y.P  )  ^  I  S(y,Y,P.)  for  all  yeYl 
’  1-1  1-1  ^  ' 

idiere 

8(x,Y,P^)  -  jjytyeY  and  ”  jjysyeY  and 

Again  the  ranking  was  based  on  the  net  scores  for  each  alternative  wltii 
the  most  positive  score  winning. 

Example  2: 

An  example  of  the  Adjusted  Borda  Method  with  N  -  5  Judges  and 
M  -  4  alternatives  is: 


Judge  1: 

w  -  X  >  y  -  z 

Judge  2: 

y  >  z  -  w  >  X 

Judge  3: 

X  >  y  -  w  >  z 

Judge  4: 

w  -  z  >  X  >  y 

Judge  5: 

y  >  w  -  X  -  z 

which  resolves  to  the  following  vote  matrix;  If  indifferences  between 
pairs  are  each  given  one-half  vote: 


Borda 

Column 

Totals 


Borda 

Row 


Alternate 

Borda 


w 

X 

y 

z 

Totals 

Totals 

3 

2  1/2 

3  1/2 

9 

3 

2 

- 

3 

2  1/2 

7  1/2 

0 

2  1/2 

2 

- 

3  1/2 

8 

1 

1  1/2 

2  1/2 

1  1/2 

- 

5  1/2 

-4 

6 

7  1/2 

7 

9  1/2 

For  the  one-half  vote  to  score,  the  Borda  aethod  of  aarfcs  technique 
had  the  Borda  Row  Totals  Coluan  with  the  Borda  aarks;  l.e.,  9,  8,  7  1/2, 
5  1/2  with  order  w  >  y  >  x  >  a.  But  with  the  Adjusted  Borda  Method, 
the  total  scores  which  were  the  row  scores  alnus  the  coluan  scores, 
were  3,  1,  0,  -4  with  order:  w  >  y  >  x  >  s.  The  Borda  and  Adjusted 
Borda  Methods,  using  one  half  votes  for  ties,  gave  the  sane  rank  orders. 

But  if  the  ties  were  calculated  as  Black  [4]  suggested  that  Borda 
prescribed  them,  then  last  ranked  tied  alternatives  In  a  Judge  list 
were  given  zero.  Those  above  the  lowest  rank,  and  tied,  were  both 
given  the  votes  to  represent  the  position  of  the  tied  pair.  Using 
the  classical  method  for  the  Borda  and  Adjusted  Borda  counts.  Example  2 
will  give  the  following: 

Borda  Adjusted- 

Row  Borda 


Borda 

Column  3667 

Totals 

By  the  original  Borda  tie  counting  method,  the  Borda  count  Is 
7,  6,  6,  3  and  the  Adjusted  Borda  count  Is  3,  1,  0,  -4.  The  corre¬ 
sponding  rank  orders  are  Borda:  y  >  w  ■  x  >  z  and  Adjusted  Borda: 
w  >  y  >  X  >  z.  The  Adjusted  Borda  methods  for  both  tie  scoring  tech¬ 
niques  gave  the  same  rank  order,  but  the  Borda  rank  orders  differed. 
Black  [4]  emphasized  that  the  Borda  method  was  Inappropriate  where 
Indifference  (ties)  existed. 
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c.  Condorcet's  Criterion 

Black  [4],  Fishbum  [119]*  and  Fisbbum  and  Gehreleln 
[129]  all  Interpreted  the  Marquis  de  Condorcet's  essay,  published  in 
1785  to  say  that,  if,  out  of  three  or  more  alternatives,  one  was  able 
to  get  a  majority  over  each  of  the  others,  it  ought  to  be  selected. 

The  iiqtllcation  was  made  that  another  method  must  be  used  if  the  one 
alternative  does  not  have  the  Condorcet  majority. 

Fishbum  [119]  defined  Condorcet's  proposal  as  a  decision  function, 

F(Y,D)  ■  C(Y,D)  whenever  C(Y,D)  existed 

where 

C(Y,D)  -  {X:XeY  and  lOty  for  all  yeY  -  (x)}  ; 

thus,  either 

C(Y,D)  *  1  or  4,  where  4  meant  a  Condorcet  winner  did  not  exist. 

This  also  proposed  that  if  an  alternative  had  a  simple  majority  over 
each  other  alternative,  then  it  should  win.  The  Condorcet  criterion 
picked  a  winner.  The  second  place  might  be  picked  by  the  same  rule 
after  the  first  place  alternative  had  been  selected  and  its  data  with¬ 
drawn.  There  was,  in  each  aggregation,  a  strong  probability  of  no  winner 
by  the  Condorcet  criterion. 

In  Example  1  for  the  Borda  Rule  Method,  the  Condorcet  evaluation 
of  the  majority  winner  of  pairs,  in  matrix  form,  would  be  as  in  the 
example  that  follows,  which  will  be  labeled  Example  3: 

Example  1  Example  3 


In  Sxaaple  3,  a  1  In  Row  w«  Coluan  x  (w»x)  neans  that,  in  Rxaaple 
1,  (w  has  a  aajorlty  over  x);  a  1  In  v,y  aeans  and  a  1  in  w,z 
neans  idls.  In  the  second  row  a  0  in  x,w  neans  not  xMw;  therefore,  w 
does  have  a  najorlty  over  all  other  alternatives  but  x  does  not.  W 
is  the  strong  Condorcet  winner.  Neglecting  w  data,  x  is  the  Condorcet 
najority  over  all  other  candidates.  The  Condorcet  rank  order  was 
w  >  X  >  y  >  2  while  the  Borda  rule  rank  order  was  x  >  w  >  y  >  z. 

The  sane  Condorcet  procedure  applied  to  Exanple  2  produced  the 
following  matrix  as  Example  4: 

Example  2  Example  4 


w 

X 

y 

z 

w 

X 

y 

z 

w 

- 

3 

2  1/2 

3  1/2 

w 

- 

1 

1/2 

1 

X 

2 

- 

3 

2  1/2 

X 

0 

- 

1 

1/2 

y 

2  1/2 

2 

- 

3  1/2 

y 

1/2 

0 

- 

1 

z 

1  1/2 

2  1/2 

1  1/2 

- 

z 

0 

1/2 

0 

- 

for  w,  wNx,  wRy,  and  ^z,  so  w  is  a  weak  Condorcet  winner.  For  x, 
the  preferences  are  not  xNw,  but  zify  and  xRz.  Therefore  x  is  a  weak 
Condorcet  second  place  in  the  rank  order.  Examples  also  can  be  shown 
where  there  are  no  Condorcet  winners. 

d.  Black's  Simple  Majority  Procedure 

Black  14]  recommended  a  procedure  where  the  Condorcet 
procedure  was  applied  first  to  determine  an  alternative  that  had  a 
najorlty  over  all  other  alternatives.  If  no  Condorcet  winner  existed. 
Black's  second  step  was  to  apply  the  Borda  nethod,  or,  if  indifferences 
existed,  the  Adjusted  Borda  nethod. 

Rlchelaon  [200]  defined  this  Black  nethod  as  the  social  decision 

rule: 
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P(Y,D)  -  {x:xeY  and  xlty  for  all  yeY} 

IFF  {x:xeY  and  xMy  for  all  yeY}  j*  0 

(n 
x:xeY  and  E  r. (x,Y,D)| 

1-1  ^  ’ 

>  I  E  rj^(y,Y,D)  for  all  yeYj  otherwise. 

e.  Dodgson's  Method  of  Inversion 

Black  [4]  Interpreted  the  1873  to  1885  writings  of  the 
Rev.  Charles  Lutwldge  Dodgson  on  committees  and  elections.  Dodgson 
also  wrote  under  the  pseudonym  Lewis  Carroll..  Dodgson  was  first  to 
use  matrix  notation  and  to  well  understand  acyclicity.  His  method  was 
that  If  Initially  there  was  no  majority  alternative,  the  decision  maker 
should  Indicate  which  alternate  would  win  by  virtue  of  the  smallest 
amount  of  change  In  the  rating  rank  orders  of  the  Judges. 

Rlchelson  [200,  202,  203]  defined  Dodgson's  procedures  as  the 
decision  rule: 

F(Y,D)  -  {x:xeY  and  t(x,Y,D)  <  t(y,Y,D)  for  all  yeY} 
where  t(x,Y,D)  Is  the  least  number  of  Inversions  needed  for  x  to  obtain 
a  simple  majority  over  every  other  alternative.  An  Inversion  occurred, 
for  example.  If  one  x,  y  In  a  preference  ordering  was  changed  to  y,  x. 
Example  5:  For  example.  If  N  -  5  judges  and  M  -  4  alternatives,  and 
Judge  1:  w>x>y>z 
Judge  2:  x>y>z>w 
Judge  3:  y>w>x>z 
Judge  4:  w>x>z>y 
Judge  5:  x>z>y>w 

the  matrices  would  be  as  follows  with  the  left  one  giving  vote  sums 
and  the  right  one  showing  pair  majority  results: 
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Exaaple  5 


w  x  y  2 

w  X  y  z 

w 

-  3  2  3 

w 

O 

X 

2-45 

X 

0-11 

y 

3  1-3 

y 

1 

O 

z 

2  0  2  - 

z 

0  0  0  -1 

Now  by  Inverting  only  one  pair  In  Judge  3's  rank  order  from  w  >  x 

to  X  >  V,  the  sum  for  the  pair  would  cause  xMw  and  alternative  x,  which 

has  a  majority  over  all  other  alternatives,  becomes  the  Dodgson  winner. 

The  second  place  Dodgson  alternative  would  be  w,  with  y  as  the  third 

place.  By  a  different  single  Inversion,  w  could  be  the  Dodgson  winner. 

f.  Copeland's  Majority  Rule  Method 

Goodman  In  Thrall  [41]  reported  the  work  of  K.  H.  Copeland, 

presented  as  mimeographed  notes  titled  "A  'Reasonable*  Social  Welfare 

Function,"  November  1951,  at  a  University  of  Michigan  Seminar  on 

Applications  of  Mathematics  to  the  Social  Sciences.  The  method,  which 

permitted  row  ordering,  was  paraphrased  by  Goodman  as: 

If  sgn  X  be  the  slgnum  function  of  x  ("slgnum"  Is  defined  from  Latin 

as  "a  sign"  or  "a  physical  representation") : 

+1  for  X  >  0 
sgn  X  ■  0  for  x  ■  0 

-1  for  X  <  0 

(x  >  0  Is  Interpreted  to  mean  that  for  an  array  of  1,  k  cells  the  row 
term  1  Is  greater  than  the  column  term  k  and  x  <  0  would  mean  k  >  1). 

Then, 

+1  for  1  >  k 
sgn  ^  ■  0  for  1  ■  k 

-1  f or  k  <  1 


where 
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«  8 

^(1)  "  £  8ga  ^(l,k)  i8  the  8core  for  row  1.  The  row  with  the 

k-1 

greateet  ^(1)  le  chosen. 

Flshbum  [17,  119]  defined  the  decision  function  as: 

F(Y,D)  -  {x:xeY  and  8(x,Y,D)  >  8(y,Y,D)  for  all  ycY} 

where 

s(x,Y,D)  ■  #{y:yeY  and  xMy}  -  #{y:yeY  and  yMx},  which  means  that 
s(x,Y,D)  Is  the  number  of  alternatives  In  Y  that  x  had  a  strict  simple 
majority  over,  minus  the  number  of  alternatives  In  Y  that  had  a  strict 
simple  majority  over  x.  Flshbum  emphasized  that  this  method  was  not 
Intended  for  weak  orders. 

Rlchelson  [200,  202,  203]  defined  the  Copeland  method  function 
the  same  way  as  Flshbum  did  with  ycUy  defined  to  be  that  x  was  preferred 
by  a  majority  to  y;  l.e.,  n(x,y}  >  n(y,x},  which  Is  a  strong  rank~order 
definition. 

By  applying  the  Copeland  method  to  Example  1,  Example  6  was 
obtained: 

Example  1  Example  6  ^ 

wxyz  wxyz  Score 

+3 
+1 
-1 
-3 

In  the  s  score  column  are  the  Copeland  scores  for  this  aggregation: 
w:4'3,  x:+l,  y:-l,  and  z:-3  idilch  gives  a  rank  order  ofw>x>y>z 
that  Is  the  same  as  the  Condorcet  rank  order  for  Example  3. 

By  definition,  the  Copeland  method  should  not  be  used  for  Example 
2  which  had  weak  rank  orders. 


w 

3 

3 

3  w  - 

+1 

+1 

+1 

X  2 

- 

5 

5  X  -1 

- 

+1 

+1 

y  2 

0 

- 

3  y  -1 

-1 

- 

+1 

z  2 

0 

2 

-  z  -1 

-1 

-1 

- 
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Flshburn  [17,  119]  presented  an  example: 


Judges  1,  2,  3, 4  y>x>a>c>b 

Judges  5, 6, 7  b>c>y>a>x 

Judges  8, 9  x>a>b>c>y 

which  has  x  as  the  Copeland  method  winner,  y  as  the  Borda  method 

winner,  and  no  Condorcet  winner.  This  is  shown  as  Example  7: 


X 

y 

a 

b 

c 

Borda 

Adj 

Borda 

X 

y 

a 

b 

c 

Cope  Ian 
Score 

X 

- 

2 

6 

6 

6 

20 

+4 

X 

- 

-1 

+1 

+1 

+1 

2 

y 

7 

- 

7 

4 

4 

22 

+8 

y 

+1 

- 

+1 

-1 

-1 

0 

a 

3 

2 

- 

6 

6 

17 

-2 

a 

-1 

-1 

- 

+1 

+1 

0 

b 

3 

5 

3 

- 

5 

16 

-4 

b 

-1 

+1 

-1 

- 

+1 

0 

c 

3 

5 

3 

4 

- 

15 

-6 

c 

-1 

+1 

-1 

-1 

- 

-2 

Col 

Totals 

16 

14 

19 

20 

21 

The  Copeland  method  rank  order  is 
x>y  =  a  =  b>c  , 
while  the  Borda  methods  rank  order  is 
y>x>a>b>c 

Flshburn  pointed  out  that  x  won  the  Copeland  method  over  y  even 
though  in  the  x,y  competition  y  had  seven  votes  and  x  had  only  two. 
g.  Shannon  and  Svestka's  Majority  Rule  Method 

Svestka  [40],  Shannon  [217],  and  Wyatt  [47]  described 
the  Shannon  and  Svestka  majority- rule  method,  which  will  be  called 
the  "Shannon  Majority-Rule  Method"  throughout  the  remainder  of  this 
dissertation.  Svestka  described  the  method  as,  first,  the  aggregation 
of  the  individual  orders  through  the  use  of  the  Kendall  array.  Next, 
the  Kendall  array  was  modified  to  form  a  second  array.  Since  the  sum 
of  two  opposing  cells  (x,y  and  y,x)  in  the  first  array  equaled  the 
number  of  judges  voting  for  the  alternatives  represented  by  that  cell 
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(x>y):  that  cell,  of  each  p  ilr,  containing  the  larger  number  of  votes 
dominated  its  opposing  cell  Therefore,  the  numbers  in  all  dominating 
cells  were  replaced  by  one  nd  the  numbers  in  all  dominated  cells  by 
zero.  The  new  row  values  wt  re  summed  into  the  a^  column.  The  rank 
order  was  given  by  decreasing  magnitudes  of  the  a^  values,  i.e.,  the 
most  preferred  had  the  largi  Jt  a^^  value. 

Shannon  (217]  and  Wyatt  (47]  described  the  method  as  first 
constructing  matrices  for  each  judge  where  each  alternative  was  compared 
to  the  others  and  a  one  was  placed  in  each  row-column  intersection 
where  the  row  alternative  wa^^  preferred  over  the  column  alternative. 

The  major  diagonal  was  disregarded.  The  judge  frequency  matrices  were 

/ 

added  to  form  a  summed  frequ«'ncy  matrix  where  a  count  was  made  for 
each  alternative  of  the  numb,  r  of  times  it  was  preferred  to  each  of 
the  other  alternatives.  In  case  of  indifference  (ties  by  a  judge) 
between  two  alternatives,  the  single  vote  was  split  with  one-half  vote 
to  each  alternative.  The  sum  of  the  complement  cells,  in  the  summed 
frequency  matrix,  even  with  ties,  equaled  the  number  of  judges  voting 
on  that  pair  of  alternatives.  Next,  the  second  matrix,  the  preference 
matrix,  was  developed  by  a  comparison  of  the  number  of  votes  in  the 
complement  cells  (x,y  versus  y,x).  If  the  x,y  value  was  greater  than 
the  y,x  value,  then  x  dominated  y  and  a  1  was  placed  in  the  x,y  cell 
and  a  zero  in  the  y,x  cell  of  the  preference  matrix.  If  the  x,y  value 
equaled  y,x,  then  each  complement  cell  in  the  matrix  was  given  one- 
half.  The  cells  in  each  row  of  the  preference  matrix  were  summed. 

The  alternatives  were  ranked  by  the  order  of  their  preference  matrix 
row  sums,  a^.  The  row  sum  figures,  a^,  were  the  number  of  times  the 
row  alternatives  had  a  majority  of  votes  over  each  of  the  other  alterna¬ 
tives. 
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In  terminology  like  Coi'ieland's,  the  Shannon  method  can  be  defined* 

I 

if  sgn  is  the  slgnum  function,  as 


sgn  (|)  ■ 


+1  for  i  >  k 
1/2  for  i  -  k 
0  for  k  >  i 


where 

r 

4>(i)  -  E  sgn  4»(l,k) 
k-1 

A 

is  the  score  of  Row  i.  The  rows  were  ranked  based  on  their  ^(i) 
magnitudes.  In  Fishburn's  and  Riche  Ison's  terminology,  the  Shannon 
method  would  be  defined  as 

F(Y,D)  =  {xrxeY  and  s(x,Y,D)  >  s(y,Y,D)  for  all  yeY} 

where 

s(x,Y,D)  *  )!f{y:yeY  and  xRy} 
which  means  that 

s(x,Y,0)  is  the  number  of  alternatives  in  Y  over  which  x  had  a 
weak  simple  majority. 

By  applying  the  Shannon  method  to  Example  1,  Example  8  was  obtained 


Sum  Frequency  Matrix 
Example  1 

W  X  V  z 


w 

X 

y 

z 


Preference  Matrix 
Example  8 
w  X  y  z 


-  3  3  3 

w 

-  Ill 

2-55 

X 

0-11 

2  0-3 

y 

0  0-1 

2  0  2  - 

z 

0  0  0  - 

Scores 

3 

2 

1 

0 


The  Shannon  a^  scores  are  in  the  s  column:  w:3,  x:2,  y:l,  z:0  which 
gives  a  rank  order  of 


w  >  X  >  y  >  z 

which  is  the  same  as  the  Condorcet  and  the  Copeland  rank  order. 
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But  unlike  the  strict  order  Copeland  method,  the  Shannon  method 
can  be  used  for  Example  2  and  will  give  the  same  preference  matrix  as 
the  matrix  In  Example  4  for  the  Condorcet  rank  order  winner.  However, 
the  Shannon  Method  will  glvi  a  rank  order  for  an  example  where  the 
Condorcet  method  might  only  give  a  first  place  winner  for  that  exaiiq;>le. 
h.  Black's  Single-Peaked  Preference 

Black  [4]  presented  a  theorem  on  single-peaked  preference 

4 

curves  of  judges  to  say  that  a  simple  majority  would  be  obtained  by  an 
alternative  if  all  judges'  preference  curves  were  single-peaked  and 
there  was  an  odd  number  of  judges.  Black  said  that  this  theorem  applied 
for  any  majority-rule  procedure.  In  definition.  Black  said  it  would 
be  possible,  when  preference  orders  were  represented  by  two-dimensional 
diagrams,  to  find  an  ordering  of  the  points  on  the  horizontal  axis 
that  left  the  curve  as  one  which  changed  Its  direction  at  most  once 
from  up  or  down.  The  key  seemed  to  be  finding  an  ordering  for  the 
horizontal  axis.  This  researcher  has  not  found  where  Black  nor  other 
researchers  have  developed  procedures  to  simplify  the  search  for  this 
one  order  that  will  permit  slngle-peakedness .  This  condilnatorlal 
problem  grows  progressively  worse  as  the  number  of  alternatives  Increases 
above  three. 

B.  Majority-Rule-Methods  Comparisons 
1.  Majority-Rule  Methods 

The  seven  majority-rule  methods  were  compared  to  determine 
If  they  satisfied  the  classification  categories  that  were  discussed  In 
Chapter  III.  The  classification  categories  were  worded  for  yes  or  no 
answers,  with  a  no  answer  preferred  for  the  purposes  of  this  research. 
Table  2  presents  the  results  of  the  comparison  of  methods.  None  of 
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the  nethods  required  only  cardinal  utilities;  therefore  this  classifi¬ 
cation  category  was  not  tabulated. 

Four  methods  have  only  one  yes  or  MOD  rating.  They  are:  b)  Adjusted 
Borda;  c),  Condorcet;  d).  Black's  Sliiq>le  Majority;  and  f),  Copeland. 

The  bj!.  Adjusted  Borda  method  decides  on  marks*  not  majorities; 

c)  Condorcet  does  not  necessarily  have  a  winner  nor  a  rank  order.  The 

d)  Black's  Simple  Majority  method*  a  combination  of  c)  Condorcet  and 
b)  Adjusted  Borda  or  a)  Borda*  Is  rated  as  a  "moderately"  difficult 
method  to  enlarge  for  numerous  Judges  and  numerous  alternatives  and 
has  either  Che  yes  of  Condorcet  or  Adjusted  Borda*  or  the  two  yes's  of 
Borda. 

Only  one  method  does  not  require  a  yes  rating.  It  Is  g)  the 
Shannon  and  SvesCka's  majority-rule  method.  Because  of  these  ratings 
and  Che  method's  flexibility  and  adaptability,  this  method  has  been 
chosen  as  the  main  thrust  of  the  aggregation  model.  Three  other 
promising  methods*  the  two  Borda 's  and  Copeland's*  will  be  Investigated 
further  during  the  research. 

2.  Aggregation  Concept  Selected 

The  Shannon  and  Svestka  Majority-Rule  Method,  described  In 
Section  A4  g  of  this  chapter,  was  chosen  as  the  basic  aggregation 
method  for  this  research.  An  appealing  advantage  of  this  method  Is 
that  cyclic  or  Intransitive  type  aggregate  rank  orders  appear  only  as 
Indifferent  alternatives. 

Example  8*  Section  A4  g,  was  an  abbreviated  example  of  the  method. 
Example  9  Is  a  more  detailed  description  of  the  Shannon  Method.  Let 
the  five  Judges*  rankings  of  four  alternatives  (w*  x*  y*  s)  be  as 
follows: 
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Example  9 


Judge  1:  w>x>y>z 
Judge  2:  x  >  z  >  y 
Judge  3:  y>wBz>z 
Judge  4:  w>y>z>x 
Judge  5:  z  >  y 

For  each  judge,  a  frequency  matrix  was  constructed  to  show  When 
each  alternative  was  preferred  over  each  other  alternate.  A  Judge  was 
alloted  one  vote  per  pair  of  alternatives;  therefore,  a  tied  pair  gave 
one-half  vote  to  each  alternative  In  the  pair.  The  alternative  for 
each  row  was  compared  to  all  column  alternatives  and  a  1  was  placed  at 
each  intersection  where  the  row  alternative  was  preferred  to  the  column 
alternative.  Disregarding  the  major  diagonal  which  is  the  intersection 
of  like  alternatives.  Judges  1,  2,  and  3  voted  indifferently  toward 
several  alternatives.  Judges  2  and  5  did  not  include  all  four  alter¬ 
natives  in  their  partial  rank  orders.  The  individual  frequency  aiatrices 
for  Example  9  are  as  follows: 


VIhen  the  five  Judges'  frequency  iMtrlces  were  sunsed*  the  result 
was  the  sunned  frequency  matrix  which  suanarlzed  all  of  the  votes  cast 


for  each  alternative. 

Next,  the  conplementary  relationship  of  each  alternative  pair 
(x,y  or  y,x)  was  compared  to  determine  which  relationship  x  >  y  or  y  >  x 
was  donlnant  for  the  summed  frequency  matrix.  In  Example  9,  for  w  and 
X,  w,x  ■  3  while  x,w  >  0;  therefore,  w  dominates  x  (%rtfx).  The  prefer¬ 
ence  was  constructed  next  giving  I's  for  the  most  preferred  (dominant) 
complement  of  a  pair  and  O's  for  the  least  preferred  (dominated) 
alternative  of  the  pair.  Note  that  z  and  x  in  the  summed  frequency 
matrix  were  tied  with  two  votes  each.  For  the  tied  relations,  such  as 
z  and  X  in  Example  9,  0.5  was  assigned  to  each  alternative  (z,x  ■  0.5 
and  x,z  •  0.5).  The  sum  of  the  majority  victory  and  tie  scores  for 
each  alternative  (each  row)  in  the  preference  matrix  provided  a  total 
preference  score.  If  the  alternatives  are  then  ranked  in  descending 
order  of  the  preference  row  sums,  a,  the  result  is  the  Shannon  Najority 
Rule  Method  rank  order,  as  in  Example  9: 

(Example  9) 

Preference  Matrix 


The  Shannon  Method  Rank  Order  isw>y>x-z 


A6 


In  sunmary,  the  Shannon  Kajorlty-Kule  Method  has  been  selected 
and  demonstrated  as  the  preferred  method  for  aggregation  of  multiple 
criteria  rank-ordered  R  &  D  projects  Into  a  single  rank-ordered  list 
of  R  &  D  projects.  The  Copeland  and  both  Borda  methods  have  had 
previous  usage  and  are  of  analytical  Interest.  Since  they  can  be 
computed  as  simple  deviations  from  the  Shannon  method^  the  Copeland 
and  both  Borda  methods  will  be  further  explored  In  this  research. 
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CHAPTER  IV.  AGGREGATION  MODEL 


In  this  chapter,  a  model  for  the  aggregation  of  multiple  ordinal 
rank  orders  Is  developed.  The  model  Includes  the  ancillary  features 
of  Input  conversions,  weighting,  fuzzy  set  orders,  and  evaluation  tests. 

A.  Assumptions 

The  model  for  the  aggregation  of  R  &  D  project  rank  orders  Into 
a  priority  list  was  based  on  several  restrictive  assumptions  as  follows: 

1)  The  number  of  judges  and  the  number  of  alternatives  both 
exceeded  two. 

2)  The  Individual  judge  ranking  actions  were  one>time  decisions, 
not  sequential  steps,  nor  revisions. 

3)  There  was  no  dialogue  or  coalition  between  the  judges. 

4)  The  set  of  individual  judge  sub lists  was  multiple  criteria 

In  that  each  judge's  rank  order  was  based  on  his  own  preference  criteria 
for  the  R  &  D  projects  or  product  requirements. 

5}  When  a  judge's  ranking  of  several  alternatives  was  Indifferent, 
It  was  a  transitive  indifference.  When  the  ranking  of  more  than  two 
alternatives  was  Indifferent,  this  was  a  restrictive  assumption. 

6)  The  individual  judge's  sublists  were  transitive  rank  orders. 
Each  list  rank  ordered  an  alternative  only  once. 

7)  The  judge's  Individual  preference  sublists  were  sincerely 
raidced.  Insincere  rank  orders  were  not  developed  to  Implement  a 
strategy  to  attempt  to  force  the  final  aggregated  answer. 
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8)  The  individual  sublists  were  ordinal  rank  orders  when 
aggregated.  If  one  list  was  originally  prepared  with  cardinal  values, 
it  was  converted  to  an  ordinal  rank  order  before  aggregation. 

B.  Model  Core  Characteristics 

The  central  core  functions  of  the  aggregation  model  were  sized 
to  aggregate  up  to  100  full  or  partial  length  individual  rank  orders 
with  a  maximum  of  100  different  alternatives. 

The  individual  sublists  were  in  a  standard  rank-ordered  format  when 
input  to  the  core.  The  standard  format  was  a  single  sequence  of  Integer 
index  numbers  which  represented  the  relative  positions  of  each  ordinal 
alternative.  The  Shannon  Majority-Rule  Method,  a  matrix  format  aggre¬ 
gation  method  described  in  Chapter  III  A,  Sections  4g  and  III  B  2,  was 
the  central  analysis  device  in  the  model  core. 

C.  Model  Ancillary  Characteristics 

A  variety  of  ancillary  features  supported  the  aggregation  model 
core.  These  features  included  input  manipulation,  weighting,  fuzzy  set 
comparisons,  and  aggregation  evaluation.  The  characteristics  of  each 
ancillary  feature  are  discussed  in  the  following  paragraphs: 

1.  Input  Manipulation 

The  Judge's  sublists  may  originally  be  prepared  as  rank 
orderings  of  R  &  D  projects  or  as  rank  orderings  of  required  products 
which  could  be  manufactured  after  completion  of  the  appropriate  R  &  D 
projects. 

a.  Projects  Sublists 

The  individual  judge's  sublists  of  R  &  D  projects  may 
be  ordered  in  several  formats  and  can  have  different  characteristics. 

The  format  can  be  hierarchical  categories  or  cardinal  lists  while  the 


list  characteristics  can  be  complete,  partial,  or  have  a  key  threshold 
alternative  separating  two  portions.  The  objective  of  the  input 
features  was  to  con  rt  the  input  projects'  sublists  into  a  single 
rank-ordered  ordinal  sublist  of  projects  for  aggregation.  For  the 
larger  problems,  the  model  with  automation  for  the  conversion  should 
significantly  reduce  errors. 

1)  Formats  -  Each  input  alternative  sublist  can  have 
any  of  the  following  formats: 

a)  Cardinal  Utility  Valued  -  The  sublist  is  input 
by  a  judge  as  a  sequence  of  cardinal  utility  values,  one  for  each  alter¬ 
native.  These  cardinal  values  could  be  performance  values,  unit  profits, 
or  any  other  values.  In  all  cases,  for  this  format,  the  values  must  be 
those  In  which  a  higher  value  denotes  a  higher  rank. 

The  model  converts  the  input  into  an  ordinal  rank  order  of 
alternative-identifying  numbers  with  the  order  determined  by  the  order 
of  the  cardinal  utility  values. 

b)  Ascending  Values  -  The  format  values  will  be 
those  in  which  a  lower  value  denotes  a  higher  rank.  The  model  converts 
the  input  subllsts  into  an  ordinal  rank  order  of  alternative  identifying 

r 

numbers  with  the  order  as  the  inverse  of  the  order  of  values.  The 
sub list  may  be  input  as  a  sequence  of  Julian  dates  or  years.  Each 
alternative  would  have  one  date.  The  dates  represent  the  time  when 
the  R  &  0  Project  will  be  completed  or  when  the  required  product  will 
be  produced  and  operational.  The  sublist  could  be  input  as  a  sequence 
of  projected  costs  for  each  project  or  required  product.  The  format 
values  will  be  those  in  which  a  lower  cost  denotes  a  higher  rank. 

This  same  input  feature  also  applies  to  a  list  that  has  other  cardinal 
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utility  values  In  which  a  small  value  denotes  a  higher  rank.  The 
model  converts  the  Input  subllst  data  Into  an  ordinal  rank  order  of 
alternative  Identifying  numbers  with  the  Index  order  as  the  Inverse 
of  the  order  of  cardinal  values. 

c)  Ranked  Categories  of  Ranked  Alternatives  - 
The  subllst  Is  Input  as  a  two-tier  hierarchical  list  In  which  the 
categories  consist  of  groupings  of  alternatives.  In  this  format,  both 
the  categories  are  ranked  and  the  alternatives  within  each  category 
are  ranked  separately.  The  Intent  Is  to  generate  a  single  rank-ordered 
list  of  alternatives.  The  category  priority  list  Is  Input  as  are  the 
priority  lists  for  the  sets  of  the  alternatives  In  each  category.  The 
model  converts  the  hierarchical  prioritized  list  and  multiple  alterna¬ 
tive  lists  into  a  single  rank-ordered  list  of  alternative  Identifying 
numbers.  The  first  alternatives  In  the  final  list  are  the  rank- 
ordered  alternatives  In  the  highest  priority  category.  These  are 
followed,  lower  on  the  list,  by  rank-ordered  alternatives  In  the 
second  priority  category.  This  process  continues  until  all  alternatives 
are  placed  In  the  final  list. 

d)  Unranked  Categories  of  Ranked  Alternatives  - 
The  subllst  Is  Input  as  a  two-tier  hierarchical  list  In  which  the 
categories  are  unranked  but  the  alternatives  within  each  category  are 
ranked  separately.  Again,  the  intent  is  to  generate  a  single  rank- 
ordered  list  of  alternatives.  The  priority  lists  for  the  set  of 
alternatives  In  each  category  are  Input.  The  model  converts  the 
hierarchical  list  and  multiple  priority  alternative  lists  Into  a  single 
rank-ordered  list  of  alternative  Identifying  numbers. 
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For  this  format,  the  categories  are  assumed  to  be  Indifferently  j 

ranked,  thereby  making  each  category  equal.  The  first  alternatives 
In  the  final  list  are  Indifferent  (equal)  listings  of  all  of  the  first 

« 

place  alternatives  In  each  category's  alternative  list.  These  are  ! 

•} 

followed,  next  lower  In  rank,  by  the  Indifferent  listing  of  all  of  the  * 

second  place  alternatives  In  each  category's  list.  This  process  1 

•  I 

continues  until  all  alternatives  are  placed  In  the  final  list.  | 

e)  Ranked  Categories  of  Unranked  Alternatives  -  j 

Again,  the  subllst  Is  Input  as  a  two-tier  hlerarchlcel  list  In  which 

the  categories  consist  of  grouping  of  alternatives.  In  this  format, 

I 

the  categories  are  ranked  but  the  alternatives  within  each  category 
are  unranked.  Again,  the  Intent  Is  to  generate  a  single  rank-ordered 
list  of  alternatives.  The  priority  list  for  the  categories  Is  Input. 

The  model  converts  the  hierarchical  prioritized  list  and  multiple 
alternative  lists  Into  a  single  rank-ordered  list  of  alternative 
Identifying  nundiers.  For  this  format,  the  alternatives  within  each 
category  are  assumed  to  be  Indifferently  ranked,  thereby  making  each 
alternative  equal  within  a  category.  The  first  alternatives  In  the 
final  list  are  the  Indifferent  (equal)  listings  of  all  of  the  altema- 

•  ; 

tlves  of  the  first  place  category.  These  are  followed,  lower  on  the 

list,  by  the  Indifferent  listing  of  all  of  the  second  place  category.  *' 

This  process  continues  until  all  alternatives  are  placed  In  the  final 

list. 

f)  Unranked  Categories  of  Unranked  Alternatives  - 
Again,  the  Input  Is  the  two-tier  subllsts  In  which  the  categories 
consist  of  groupings  of  alternatives.  In  this  format,  neither  the 
categories  nor  the  alternatives  are  ranked.  Only  Indifferent  lists 
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are  Input.  The  model  converts  the  hierarchical  lists  into  a  siiyle 
rank-ordered  list  of  alternative  identifying  nu^ers.  For  this  format, 
the  categories  are  assumed  to  be  indifferently  ranked,  thereby  making 
each  category  equal.  Also,  the  alternatives  within  each  category  are 
assumed  to  be  indifferently  ranked,  thereby  making  each  alternative 
equal  within  a  category.  The  alternatives  in  the  final  list  are  the 
equal  listing  of  all  alternatives  in  all  categories,  since  the  input 
gives  no  information  for  any  relative  differentiation. 

2)  Characteristics  -  The  input  sublists  of  alter¬ 
natives  can  also  have  one  of  the  following  characteristics: 

a)  Complete  Sublist  -  The  sublist  is  input  as  a 
rank-ordered  sequence  of  alternative  identifications  which  ranks  all 
alternatives  of  Interest  to  the  aggregation  problem  in  question.  The 
model  core  accepts  this  sublist  in  its  initial  form. 

b)  Incomplete  Sublist  -  The  sublist  input  ranks 
less  than  all  alternatives  of  interest  to  the  aggregation  problem. 

The  model  may  accept  the  Incomplete  sublist,  as  is,  for  aggregation,  or, 
upon  command,  the  model  may  synthetically  complete  the  sublist.  The 
model  first  determines  which  alternatives  were  not  included  in  the 
input  sublist.  The  judge  is  assumed  to  be  indifferent  (equal)  concerning 
the  relative  relationship  of  alternatives  not  Included  in  the  input. 

Then,  as  commanded,  the  model  adds  the  alternatives  not  Included  as 
Inputs  equally,  at  one  level  lower  rank,  to  create  the  synthetically 
completed  rank  order.  If  otherwise  commanded,  the  alternatives  not 
Included  as  input  can  be  added  equally,  at  one  level  higher  rank,  to 
create  the  beginning  of  the  synthetically  completed  order.  These  two 
completion  alternatives  represent  the  situation  where  the  input 
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Incomplete  alternative's  sublist  is  rated  either  as  better  than  all 
of  those  alternatives  not  included  or  as  worse  than  all  of  those  alter¬ 
natives  not  included. 

c)  Threshold  List  -  The  sublist  is  input  as  an 
incomplete  rank-ordered  sequence  of  alternative  identifications.  Also, 
input  is  a  specific  threshold  key  alternative  and  an  incomplete 
reference  rank-ordered  list.  The  model  will  merge  the  input  sublist 
with  the  reference  list  using  the  rank  policy  that  all  of  the  input 
list  is  preferred  to  the  key  threshold  alternative  in  the  reference 
list  and  all  alternatives  below  the  key  alternative.  By  command,  the 
direction  may  be  reversed  where  all  of  the  input  list  is  designated 
as  Inferior  to  the  key  threshold  alternative  in  the  reference  list  and 
all  alternatives  above  the  key  alternatives.  To  avoid  Intransltlvites, 
the  combined  list  is  searched  for  any  duplicate  occurrences  of  the 
same  alternative.  The  assumption  is  made  that  if  duplicates  occur  in 
a  rank  order,  the  higher  ranked  occurrence  of  an  alternative  will 
prevail.  Therefore,  the  model  eliminates  the  lower  ranked  of  all 
duplicates  found  in  the  combined  list.  The  final  list  is  a  sequence 
of  nonduplicating  alternative  identification  numbers, 
b.  Requirements  Sublist. 

The  individual  judges  may  rank  order  lists  of  require¬ 
ments  for  products  which  cause  R  &  D  projects.  Instead  of  being  the 
R  &  D  projects.  There  are  not  necessarily  one-to-one  relations 
between  requirements  and  projects.  One  project  may  directly  benefit 
several  products  or  one  product  requirement  may  necessitate  projects 
in  several  areas  of  R  &  D. 
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1)  ,  Formats  and  Characteristics  -  All  of  the  Input 
formats  and  characteristics  of  this  Chapter,  Section  C,  for  R  &  D 
projects  apply  equally  for  requirements.  The  alternatives  In  the 
Inputs  can  represent  projects  or  requirements.  |  The  model  Is  designed 

to  accomplish  all  forms  of  Input  conversions  for  projects  Independently 

« 

of  requirements.  But  before  the  model  can  aggregate  project  rank 
orders^  all  requirement  alternatives  must  be  translated  to  projects. 

2)  Translation  to  Projects  Subllsts  >  Requirement 
alternatives  are  translated  to  project  alternatives  through  a  trans*> 
latlon  Index  in  the  model.  The  translation  Index  file  consists  of 
equivalency  statements  between  requirements  ‘and  projects.  When  one 
requirement  is  equivalent  to  more  than  one  project,  the  several  Joint 
projects  are  each  assumed  to  be  equally  related  to  the  requirement. 
Also,  when  several  requirements  are  Jointly  equivalent  to  one  project, 
the  several  Joint  requirements  are  each  assumed* to  be  equally  related 
to  the  project. 

The  model  steps  througli  the  requirement  alternative  In  a  converted 
Input  sublist  one-by-one.  Each  project  in  the  translation  index 
replaces  Its  equivalent  requirement  In  the  rank-ordered  sublist.  Where 
multiple  equivalency  occurs,  several  projects  equally  replace  one 
requirement.  A  new  project  rank  order  Is  thus  constructed.  But  this 
new  project  rank  order  probably  has  duplicates,  triplicates,  etc.,  of 
the  same  project  alternative.  Again,  the  assumption  Is  used  that  an 
R  &  D  project's  priority  level  should  be  derived  from  the  highest 
level  at  which  It  is  required.  Therefore,  the  model  locates  all 
replications  of  each  alternative  and  eliminates  all  but  the  highest 


occurrence  of  each  alternative.  This  eliminates  Intransitive  inputs 
Into  the  aggregation  matrix. 

Due  to  the  multiplicity  in  the  translation  Index,  the  number  of 
projects  In  a  translated  rank-ordered  subllst  is  unknown  until  after 
each  list  has  been  translated  and  replications  eliminated.  When 
completion  of  Incomplete  input  lists  is  required,  this  Is  accomplished 
In  the  model  for  requirements  before  the  translation  step.  This 
sequence  of  completion,  then  translation,  is  derived  from  the  initial 
model  assumption  concerning  the  Independence  of  each  Judge.  In  the 
laboratory  director  scenario,  the  advisors  are  Independent  and  frequently 
submit  their  rank-ordered  sublists  by  mall.  Also,  the  translation 
Index  equivalences  probably  will  be  developed  by  laboratory  R  &  D 
managers  after  comparisons  of  the  requirements  with  technology  assess¬ 
ments.  An  advisor  (Judge)  who  ranks  requirements  has  only  an  Indirect 
relation  to  the  R  &  D  projects.  Therefore,  if  his  sublists  are  incomplete, 
they  should  be  only  synthetically  completed  by  the  remainder  of  the 
product  requirements  considered  for  the  aggregation. 

2.  Weighting  Methods 

The  relative  Importance  and  authenticity  of  each  sublist  and/or 
alternative  may  be  Included  in  the  aggregation  model  by  one  or  more 
weight  factors  applied  to  the  rank-ordered  sublists  during  the  aggregation 
process.  Two  general  types  of  weight  application  methods  are  In  the 
model.  The  decision-maker  method  applies  weight  factors  quantified  by 
the  decision  maker  responsible  for  the  aggregation.  The  Judge  self- 
evaluation  method  applies  weight  factors  quantified  by  each  of  the 
Judges . 
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a.  Declsion-Haker  Weighting  Methods 

The  aggregation  model  has  a  variety  of  weighting  mecha- 
nisms  available  to  the  decision  maker.  Weights  can  be  applied  to  each 
alternative  as  and/or  to  the  rankings  of  each  judge  as  .  The  same 
value  is  applied  to  a  specific  alternative  every  time  it  appears  in 
the  frequency  matrices.  The  factors,  and/or  w^,  are  applied  as 
multiplicative  linear  weighting  factors,  exponential  weighting  factors, 
additive  weighting  factors,  or  logarithmic  weighting  factors.  Before 
applying  any  specific  dec is ion-maker  weight,  the  1/2  tie-generated 

4 

frequency  scores  and  the  1  frequency  scores  are  normalized  by  multiplying 
all  judge  frequency  matrices  elements  by  a  uniform  4  factor.  This 
uniformly  applied  4  Inflates  the  vote  counts,  but  eases  computation  by 
eliminating  complications  caused  by  exponentials  of  fractions  reducing 
all  values.  Without  the  4  bias  factor,  the  logarithmic  weight  default 
value  of  1  would  drive  0  cell  values  into  an  undefined  status.  This 
normalization  is  beneficial  for  the  exponential  and  logarithmic  weights. 
For  the  Shannon  Majority-Rule  method,  which  scores  pair  majorities, 
this  constant  4  permits  w^  and  w^  >  1  to  be  uniformly  applied.  The 
decision-maker  weighting  methods  in  the  model  are  to  follow. 

The  choice  of  a  weighting  method  is  a  subjective  choice  of  the 
decision  maker.  The  multiplicative  and  exponential  methods  are  expected 

i 

I  to  beused  most  often.  Only  one  method  is  used  for  one  aggregation. 

1)  Alternative  Multiplicative  Weights  -  This  is  a 
linear  multiplicative  method  where  each  alternative,  i,  is  assigned  a 
weight  value,  w^.  The  w^  value  is  multiplied  by  the  1  th  alternative 
entries  in  the  judges'  frequency  matrices  in  the  Shannon  Majority-Rule 
method.  If  x^^  is  the  cell  value  in  the  judge  frequency  matrix  at  the 
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intersection  of  the  i  th  row  and  the  k  th  column,  then  the  weighted  value 

for  X..  becomes  X.!  ■  W.X^,  . 
ik  ik  i  ik 

2)  Condiined  Judge  and  Alternative  Multiplicative 
Weights  -  This  is  a  linear  multiplicative  method  with  dual  fieighting 
factors.  For  one  factor,  each  alternative,  i,  is  assigned  a  weight 
value,  w^.  For  the  second  factor,  each  Judge's  total  sublist  of  alter¬ 
natives  are  assigned  another  might,  w^.  The  w^  and  w^  values  are  both 
multiplied  by  the  1  th  alternative  entries  in  the  judges'  frequency 
matrices.  If  is  the  cell  unweighted  value  for  the  i,k  intersection, 
the  freighted  value,  becomes 


*lk  *  "j"i*lk 


3)  Alternative  Exponential  Weights  -  This  is  an  exponen¬ 


tial  uethod  where  each  alternative,  i,  is  assigned  a  weight  value,  w^. 


The  w^  value  is  applied  as  the  exponential  power  of  the  i  th  alternative 


entries  in  the  judges'  frequency  matrices.  If  x^^^  is  the  cell  value 


for  the  l,k  intersection,  the  weighted  value,  x^^,  becomes 


x;,  -  <x,,  )''i 


^ik 


•ik" 


4)  Combined  Judge  and  Alternative  Exponential  Weights  - 
This  is  an  exponential  method  with  dual  wei^tlng  factors.  Each 
alternative,  1,  is  assigned  weight  value,  w^,  and  each  judge's  total 
sublist  alternatives  are  assigned  another  factor,  w^.  The  w^  and  the 
Wj  values  are  jointly  applied  as  exponential  powers  to  the  i  th  alter¬ 


natives  in  the  judges'  frequency  matrices.  If  x^^  is  the  cell  value 


for  the  l,k  intersection,  the  weighted  value,  x^j^,  becfxnes 


x'  -  (x 
*ik  '*ik'  ^  ^ 
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5)  Combined  Alternative  Multiplicative  and  Judge 

I 

Exponential  Weights  -  This  is  a  multiplicative  and  exponential  method 
with  dual  weighting  factors.  Each  alternative,  1,  is  assigned  weight 
factor,  w^,  and  each  judge's  total  sublist  alternatives  are  assigned 
axu>ther  weight,  w^.  The  w^  and  the  Wj  values  are  jointly  applied  with 
the  w^  as  multiplicative  and  the  w^  as  an  exponential  power  to  the  i  th 
alternative  in  the  judges'  frequency  matrices.  If  x^^  is  the  cell 
value  for  the  i,k  Intersection,  the  weighted  value,  becomes 


*ik  "  "i^*ik^  ^ 

6)  Combined  Alternative  Exponential  and  Judge  Multi¬ 
plicative  Weights  -  This  is  a  multiplicative  and  exponential  method 
with  dual  weighting  factors.  Each  alternative,  i,  is  assigned  weight 
factor,  w^,  and  each  judge's  total  sublist  alternatives  are  assigned 
Wj.  The  w^  and  w^  values  are  jointly  applied,  with  the  Wj  as  multi¬ 
plicative  and  the  w^  as  an  exponential  power  to  the  i  th  alternative 
in  the  judges'  frequency  matrices.  If  x^^  is  the  cell  value  for  the 
l,k  intersection,  the  weighted  value,  x^^,  becomes 

7)  Combined  Alternative  and  Judge  Additive  Weights  - 
This  is  an  additive  method  with  dual  weighting  factors.  Each  alter¬ 
native,  1,  is  assigned  welghtli^  factor  w^,  and  each  judge's  total 
sublist  alternatives  are  assigned  w^.  The  w^  and  w^  values  are  jointly 
applied  as  additions  to  the  i  th  alternatives  in  the  judges'  frequency 
matrices.  If  x^j^  is  the  cell  value  for  the  l,k  intersection,  the 
weighted  value,  x^j^,  becomes 

■‘ik  ■  ^ik  +  "i  +  "j 
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8}  Conblned  Alternative  and  Judge  Logarithalc  Weights  - 
This  Is  a  logarithmic  method  with  dual  multiplicative  velghtlng  factors. 
Each  alternative,  1,  Is  assigned  weighting  factor  and  each  Judge's  * 
total  subllst  alternatives  are  assigned  w, .  The  w.  and  w.  values  are 

•  j  1  j  • 

jointly  applied  as  multiplicative  to  the  1  th  alternatives  In  the 
judges'  frequency  matrices.  Then  the  logarithm  Is  taken,  and  Wj 

are  restricted  to  values  equal' to  or  greater  than  one.  If  Is  the 
cell  value  for  the  l,k  Intersection,  the  weighted  value,  x^^,  becomes 

*ik  ■  i”* 

b.  Judge  Self-Evaluation  Weighting  Methods 

The  quality  of  a  rank-ordered  priority  aggregation  can 
be  significantly  influenced  by  the  variation  in  each  judge's  knowledge 
of  each  alternative  he  chooses  to  rank.  The  aggregation  model  for  this 
research  has  provisions  for  each  judge  to  rate  his  own  expertise  about 
each  alternative  in  regard  to  the  criteria  being  used  for  the  ranking. 

The  self-rating  scale  may  be  selected  differently  for  each  sublist. 
The  scale  is  restricted  In  that  the  poorest  rating  must  be  zero.  The 
best  rating  may  be  any  real  number,  except  zero.  Each  judge,  when 
ranking  the  alternatives,  also  assigns  a  self-rating  value  which  repre¬ 
sents  an  estimate  of  that  judge's  expertise  on  that  alternative  with 
regard  to  that  judge's  ranking  criteria.  The  model  converts  the  scale 
used  to  a  0  -  1  scale.  The  converted  self-expertise  rating  Is  multi¬ 
plied  by  the  alternative  scores  for  the  judge's  frequency  matrix  as  If 
the  self-rating  were  a  multiplicative  weight. 

The  model  also  provides  for  those  aggregation  problems  where  the 
decision  maker  chooses  to  disregard  alternative  rankings  by  judges  who 
have  low  self-evaluation  ratings.  The  decision  maker  chooses  a  threshold 
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rating*  on  tHe  0-1  scale.  All  Judged  alternatives  with  converted 
self-ratings  below  the  threshold  value  are  purged  from  the  Judge  sublists. 
The  aggregation  problem  continues  with  the  alternatives  self-rated  above 
the  threshold.  The  remaining  calculations  are  those  of  aggregating 
partial  sublists  with  multiplicative  weightings  for  each  alternative. 

3.  Preference  Scoring  Constants 

r 

Most  of  the  majority-rule  methods  described  in  Chapter  III* 
Section  A4  and  the  other  literature  surveyed  can  be  implemented  by 
similar  array  type  computations.  But  the  preference  scoring  constants 
of  the  methods  differ.  For  example*  the  Shannon  Majority-Rule  Method 
uses 

(  1  if  i  >  k 

x(l,k)  -  {  1/2  if  1  -  k 
I  0  if  k  >  1 

for  the  frequency  matrix  and 
(l  if  iMk 

x(i*k)  -  {  1/2  if  i  -  k 
lo  if  kMi 

for  the  preference  matrix.  On  the  other  hand*  the  Copeland  Method 
uses 


x(l*k) 


1  if  i  >  k 
0  if  k  >  i 


for  the  frequency  matrix  (ties  not  permitted)  and 

(1  if  iMk 
0  if  i  -  k 
-1  if  kMl 


for  the  preference  matrix.  Table  3  presents  the  matrix  scoring  constants 
for  the  seven  majority-rule  methods  described  in  Chapter  III*  Section  A4. 

To  permit  correlation  with  literature  examples  and  to  allow  studies 
of  the  effect  of  the  scoring  constants*  the  model  is  capable  of  aggre¬ 
gating  the  rank  orders  with  any  of  the  four  options: 


61 


Majority 

Rule  Method 

1. 

Borda 

2. 

Adjusted 

Borda 

3. 

Condorcet 

4. 

Black's  Simple 
Majority 

5. 

Dodgson 

6. 

Copeland 

7. 

Shannon 

Scoring 

1 

Constants 

Frequency  Matrix 

Preference  Matrix 

N/A  N/A  N/A 
N/A  N/A  N/A 


1/2 
0 

1/2  I  0 


Note:  N/A  denotes  that  the  constant  for  the  stated  conditions 
is  not  applicable. 


4.  FUzzy  Set  Rank  Orders 
a .  Background 

The  literature  survey  for  this  research  disclosed  that 
a  new  branch  of  social  choice  theory  has  been  emerging  since  1974.  Blln 
[70,  71],  in  1974,  proposed  that  fuzzy  set  theory  be  applied  to  the 
social  choice  problem  of  determining  group  preference.  Blin  [71]  said 
that  in  a  fuzzy  set  problem,  miltiple  observers'  opinions  are  pooled 
and  somehow  aggregated  to  reach  a  consensus  over  some  well-specified 
event.  The  model  for  this  research  had  a  limited  capability  to  study 
the  fuzzy  set  rank  orders  that  could  be  obtained  from  the  summed 


62 


i 


frequency  matrix  of  the  Shannon  Majority-Rule  method.  Where  permitted  . 
by  the  problems,  the  aggregate  fuzzy  set  rank  order  will  be  computed 
and  compared  with  the  Shannon  method  preference  matrix  rank  orders. 

Blln  [70]  explained  that  strict  fuzzy  pair  preferences  could  be 
assigned  a  value  of  1.  The  strict  reciprocal  preference  would  be 
assigned  a  0.  Those  less  strict  would  be  assigned  preference  fuzzy 
values  between  0  and  1.  When  n  Individual  fuzzy  matrices  were  summed, 
the  resulting  value  In  the  cell  Is  nk  where  k  fell  between  0  and  1. 

The  cells  In  the  Shannon  Method's  summed  frequency  matrix  had 
similar  values  which  could  be  written  as  nk  where  n  Is  the  number  of 
judge  frequency  matrices  summed  and  k  Is  an  average  score  for  the 
judges'  preference  with  k  between  0  and  1.  Therefore,  In  this  model, 
the  Shannon  Method  summed  frequency  matrix  divided  by  the  total  number 
of  judges  became  the  fuzzy  set  matrix, 
b.  Analytical  Method 

The  fundamental  definitions  of  a  fuzzy  rank  relation  R 
were  clearly  summarized  by  Buckles  [83].  He  pointed  out  that  for 


complementary  cells,  x^^  and  additional  fuzzy  set  requirement 

was  that  x . ,  -  1  -  x . . .  For  the  fuzzy  matrix  complement  cell  values  to 
ij  ji 

sum  to  1,  the  Shannon  method  rank-order  subllsts  must  be  complete  lists 


of  unweighted  alternatives.  Also,  Buckles  presented  the  fuzzy  set 


difference  definition.  In  which  the  fuzzy  set  matrix  Is  R  and  Its 
T 

transpose  Is  R  .  The  difference  matrix  definition  Is 

„  _  „T  _  /u(x,y)  -  u(y,x).  If  u(x,y)  >  u(y,x) 

\  0  otherwise 


where  for  u(x,y)  quantities^  x  Is  preferred  to  y. 

To  obtain  fuzzy  preferences,  the  steps  are  as  follows  [83]: 

T 

Step  1:  Find  the  set  difference,  R  -  R  . 
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Step  2:  Determine  the  portion  of  each  alternative  that  is 


not  dominated. 

A  "  ^  ^*1,  Col  A’  *2,  Col  A’*'**n,  Col  A^ 

tdilch  means  the  nondomlnated  value  for  an  alternative,  a,  in  the 
T 

(R  -  R  )  matrix  is  equal  to  one  minus  the  greater  of  the  values  in 
Column  A. 

Step  3:  The  rank  order  of  the  fuzzy  set  is  then  the  rank  order 
ND 

of  the  X  values  in  descending  order.  As  an  example,  let  R,  the  fuzzy 
set  matrix  be 


'O  0.9  0.2‘ 

0.1  0  0.7 

_0.8  0.3  0 


The  transpose  would  be 


0  0.1  O.Sl 

0.9  0  0.3  . 

0.2  0.7  0  J 


And  the  difference  set  would  be 


T 

R-R  - 


ro  0.9  0.2l  To  0.1  0.81  fO  0.8  0  1 

0.1  0  0.7  -  0.9  0  0.3  -  0  0  0.4  . 

Lo.8  0.3  0  J  [0.2  0.7  0  J  [o.S  0  0  J 


Then  the  nondomlnated  values  become 


=  1  -  max  (0.6)  -  1  -  0.6  =  0.4 

-  1  -  max  (0.8)  -  1  -  0.8  -  0.2 

X^^  -  1  -  max  (0.4)  -  1  -  0.4  -  0.6 


The  rank  order  for  the  fuzzy  set  values  is  then 
0.6  >  0.4  >  0.2  and  the  order  is  3  >  1  >  2. 

Buckles  [83]  summarized  Bezdek,  et  al.  [60]  scalar  measures  of 
fuzzy  set  matrices,  R.  The  two  measures  are  the  average  fuzziness 
in  R,  F(R),  and  the  average  certainty  in  R,  C(R).  The  average  fussiness 


F(R) ,  was  proportional  to  R's  fuzziness  or  uncertainty  about  pairwise 
rankings.  The  average  certainty  In  R«  C(R),  averaged  the  Individual 
dominance  of  each  district  pair  of  rankings  and  was  proportional  to  the 
overall  certainty  In  matrix  R.  The  equations  for  F(R)  and  C(R)  are: 


F(R) 


tr(R^) 


2 


2  2 
where  tr(R  )  Is  the  trace  of  the  matrix  R 


C(R) 


tr(RR^) 

n(n-l) 

2 


T  T 

idler e  tr(RR  )  Is  the  trace  of  the  matrix  (RR  ).  Also, 


F(R)  +  C(R)  =  1 


In  summary,  for  complete  aggregation  problems,  the  model  calculated 
the  fuzzy  matrix,  the  fuzzy  set  rank  order,  and  the  scalar  measures  F(R) 
and  C(R) .  But  the  model  did  not  have  the  sensitivity  recommended  by 
Bezdek  [60].  Bezdek  said  that  each  judge  should  make  each  vote  fuzzy. 
Instead  the  model  used  the  total  vote  for  an  alternative  as  fuzzy.  Only 
the  judge  self-evaluation  model  option  provided  single-vote  fuzziness. 

5 .  Aggregation  Evaluations 

The  evaluation  methods  selected  had  to  accommodate  the  several 
(n  ^  3)  rank  orders  being  aggregated,  and  the  several  aggregated  rank 
order  outputs  from  the  different  methods.  Three  Kendall  [22]  methods 
were  chosen  for  the  evaluations: 

a)  Kendall's  Coefficient  of  Concordance  Test. 

b)  Kendall's  Circular  Triads  Analysis. 

c)  Kendall's  Rank  Order  Consistency  Analysis  and  Test. 

The  Implementation  of  each  Into  the  model  will  be  discussed  In  the  next 


sections. 


a.  Coefficient  of  Concordance  Test 

1)  The  Coefficient  -  Kendall's  Coefficient  of  Concordance, 
W,  was  chosen  as  a  measure  of  the  relation  among  several  rankings  (n  ^  3) 
of  alternatives.  Arrow  [1]  said  that  Kendall's  statistic  W  may  be  used 
In  the  same  situation  in  tdilch  Friedman's  [132]  two-way  analysis  of 
variance  by  ranks  test  statistic  was  available.  Conover  [9]  said, 
further,  that  Kendall's  W  was  probably  Intended  as  a  measure  of  agreement 
In  rankings  rather  than  as  a  test  statistic.  This  Interpretation  of 
Kendall's  W  coincided  with  the  needs  of  this  research  model.  The  coeffi¬ 
cient  of  concordance  was  applied  In  two  ways.  First,  the  method  measured 
the  agreement  among  the  judges'  sublists.  Second,  the  method  measured 
the  agreement  between  combinations  of  aggregated  rank  orders  obtained 
with  different  majority-rule  methods,  l.e.,  Borda,  Adjusted  Borda,  Fuzzy 
Set,  and  Shannon.  The  first  application  provided  an  Indication  of  the 
agreement  and  divergence  of  the  judges,  while  the  second  application 
provided  a  measure  of  the  agreement  between  the  final  rank  order  results, 
not  just  the  winning  alternative,  from  different  majority-rule  methods. 

The  Kendall's  W  method  was  limited  to  sets  of  rank  orders  that  had  the 
same  length.  Therefore,  if  partial  and  complete  rank  orders  were  aggre¬ 
gated  together,  a  Kendall's  W  measurement  cannot  be  calculated  unless 
the  Incomplete  rank  orders  were  synthetically  completed. 

The  rationale  of  the  coefficient  of  concordance,  W,  was  to  serve 
as  an  index  of  the  divergence  of  the  actual  agreement  shown  In  the  data 
from  the  most  perfect  agreement  [Selgel,  37]. 

To  compute  W,  first  array  the  rank  orders  In  a  table,  with  the  M 
judge  ranks  listed  In  rows  and  alternatives  in  columns.  Next  find  the 
sum  of  the  ranks,  Rj,  in  each  alternative  column.  The  mean  sum,  Rj,  Is 
calculated  by  suamilng  the  Rj  values  for  all  alternatives  and  then 


dividing  by  the  nundier,  N,  of  different  alternatives.  Next,  the  devia¬ 
tion  from  the  mean  |  Is  calculated  for  each  alternative.  Then 
the  square  of  these  R^  deviations  Is  summed  Into  an  S  value.  S,  there¬ 
fore,  Is  stated  as  , 


Tied  alternatives  In  a  ranking  caused  complications  In  Kendall's 
W  computations.  Excess  numbers  of  tied  ranks  In  an  aggregation  tended 
to  depress  the  value  of  W.  A  correction  was  available,  Kendall  [22],  to 
adjust  this  effect  of  excessive  tied  rankings. 

The  tied  ranking  correction  and  the  squared  sum  of  deviations,  S, 
are  used  In  the  coefficient  of  concordance,  W,  equation 


1  \  2  3  ” 

M  (N  -  N)  -  M  r  T 
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where  M  Is  the  nuoiber  of  judges  and  N  Is  the  number  of  alternatives. 
If  there  are  no  significant  tied  ranks,  W  is 


2)  The  Test  -  Kendall  [22]  developed  methods  and 
special  small  N  value  probability  tables  to  test  the  hypothesis 
there  was  perfect  disagreement  between  the  judges  (there  was  no  concordance 
between  judges) .  The  test  for  varied  depending  on  the  value  of  N 
(the  number  of  alternatives).  W  varied  from  0  to  1.  It  would  be  1  when 
the  ranks  assigned  by  each  judge  were  exactly  the  same  as  those  by  other 
judges.  W  would  be  0  when  there  was  maximum  disagreement  among  the 
judges.  The  test  methods  also  varied  with  the  values  of  M.  The  method 
steps  and  tables  are  In  Dobbins  [14]. 


b.  Circular  Triads  Analysis 

Kendall's  Circular  Triads  Analysis  [22]  was  chosen  as  a 
measure  of  the  acycllty  of  the  pair  majorities^ In  the  preference  matrix 
of  the  Shannon  method.  In  preference  matrices  of  more  than  three  alter¬ 
natives,  It  was  possible  to  have  the  majority  preferences  of  three 
alternatives  aligned  to  be  circular  triads.  For  an  example,  XonHaii 
presented  a  preference  matrix  example  ([22],  pp.  145)  which  would  have 
a  Shannon  majority-rule  aggregate  rank  order  ofA-C>B>E*F>D. 
When  analyzed  Internally,  It  had  five  circular  triads:  ACDA,  ABDA,  AEDA, 
AFDA,  and  BEFB.  Triads  were  counted  because,  for  example,  any  circular 
tetrads  must  contain  two  circular  triads.  Kendall  further  proved 
that  the  maxiimim  possible  number  of  circular  triads  Is 


0^  -  N) 
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if  N  (nuiil>er  of  alternatives)  is  odd. 


and  It  Is 
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The  minimum  number  of  triads  Is  zero.  He  further  proved  that  the 
and  minimum  number  of  triads  can  be  attained  by  arrangement  of  preferences. 
Kendall's  equation  for  d,  the  nuid>er  of  circular  triads  in  a  preference 
matrix,  consisted  of  the  terns  N  (nuid>er  of  alternatives)  and  a^,  the 
sum  of  the  rows  of  the  preference  matrix.  The  equation  for  d  is 


d  -  r  N  (N  -  1)  (N  -  2) 
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The  Kendall  derivation  of  d  was  based  on  rank  orders  without  tied 
pairs  (Indifference).  When  a  preference  matrix  had  tied  pairs.  It 
caused  pairs  of  a^  terms  that  have  fractions.  The  fractions  were  always 
one-half,  l.e.,  a^  Is  1.5,  3.5,  6.5,  7.5.  When  tied  pairs  existed,  the 


68 


sum  of  the  was  not  necessarily 

f  2  \  _ N! _ ^  N  (  M  -  1) 

\2j  \2,  N  -  2/  21  (N  -  2)!  2 

which  is  the  sum  of  a^  for  integer  valued,  no  tied  pairs,  preference 
matrices.  To  resolve  this  problem,  the  model  for  this  research 
bracketed  the  possible  d  values  if  fractional  pairs  of  a^^'s  exist.  The 
steps  of  the  d  bracketing  method  are: 

Step  1:  Arrange  the  a^  row  totals  in  order  of  their  value. 

Step  2:  Count  the  number  of  fractional  a^  row  totals. 

Step  3:  Round  the  upper  one-half  of  each  pair  of  the  fractional 
a^  values  upward  to  their  next  larger  integer  values. 

Step  4:  Round  the  lower  one-half  of  each  pair  of  fractional 
a^  values  downward  to  their  next  smaller  integer. 

Step  5:  Verify  that  the  sum  of  the  rounded  equals 

1/2  N  (N  -  1). 

Step  6:  Calculate  a  d  value  for  this  rounded  set  of  a^^  values. 
Label  this  the  "lower  d"  since  it  will  give  the  lower  value  of  seta, 
the  coefficient  of  consistency  yet  to  be  described. 

Step  7:  Return  to  the  ordered  unrounded  a^'s  and  round  the  upper 
one-half  of  each  pair  of  the  fractional  a^  values  downward  to  their 
next  smaller  Integer  values. 

Step  8:  Round  the  lower  one-half  of  each  pair  of  the  fractional 
a^  values  upward  to  their  next  larger  Integer  values. 

Step  9:  Verify  that  the  sum  of  the  second  rounded  a^'s  equals 
1/2  N  (N  -  1) . 

Step  10:  Calculate  a  d  value  for  this  second  rounded  set  of  a^ 
values  and  label  this  the  "upper  d." 
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Step  11:  Average  the  "lower  d"  and  upper  d"  to  fom  an  approKlaate 
d  for  the  matrix  with  the  tied  palra.  The  approximate  d  valuea  are  not 
neceasarlly  Integera,  but  nay  be  rounded  to  an  integer, 
c.  Coefficient  of  Conalstency 

Kendall  [22]  extended  the  number  of  circular  triad 
analyses  to  a  coefficient  of  consiatency,  seta»  which  related  the 
calculated  number  of  circular  triads »  d,  to  the  — number  possible: 
1/24  (N^  -  N)  if  N  is  odd  of  1/24  -  4N)  if  N  is  even.  The  equation 

for  the  coefficient  of  consistency  is 


seta  > 


1  -  ,  if  N  is  odd 

IT  -  N 


1 - = -  t  if  M  is  even. 

IT  -  4N 


For  no  inconsistencies  (no  circular  triads) ,  seta  is  unity.  As  the 

number  of  circular  triads  increases »  seta  approaches  sero. 

The  test  method  for  Kendall's  coefficient  of  consistency  varies 

with  N.  Special  tables  modified  from  Svestka  [40]  and  Kendall  [22] » 

are  in  Dobbins  [14].  The  hypothesis  tested  is  H  :  there  is  no  consis- 

o 

tency  in  the  aggregated  rank  order.  The  test  for  H  again  varied 

o 

depending  on  the  value  of  N  (the  number  of  alternatives).  Begin  with 
a  calculated  zeta,  and  follow  the  procedures  in  Dobbins  [14]. 

D.  The  Model 

The  generalized  model  flow  to  perform  the  rank-ordered  prioritized 
list  aggregation  and  analysis  for  this  research  is  Illustrated  in 
Figure  2.  The  basic  steps  in  the  process  are  as  follows: 

1)  The  aggregation  problem  is  defined,  formulated,  and  input 


into  the  computer  code  in  a  form  c(mslstent  with  the  model 


2)  The  ctaqputer  program  converts  ell  rank-ordered  sublists  into 
its  standard  format. 

3)  Each  rank-ordered  sublist  Is  formed  Into  sublist  frequency 
matrices. 

4)  The  standardised  sublists  are  analyzed  to  determine  and 
significance  test  their  Kendall's  concordance,  W. 

5)  The  sublist  frequency  matrices  are  summed. 

6)  The  row  and  column  totals  of  the  sumsied  frequency  matrix  are 
used  to  compute  the  Borda  and  Adjusted  Borda  rank  orders. 

7)  Compute  the  preference  matrix  from  the  suimed  frequency  matrix. 

8)  Compute  the  Shannon  method  or  Copeland  method  output  rank 
order  from  the  sum  of  the  rows  of  the  preference  matrix. 

9)  Compute  and  test  the  circular  triads  and  consistency  of  the 
preference  matrix. 

10)  Compute  the  fuzzy  matrix  and  fuzzy  rank  order  from  the  summed 
frequency  matrix. 

The  preceding  generalized  model  was  used  to  develop  the  model 
explained  in  the  following  two  chapters  and  used  for  the  R  &  D  Project 
Prioritization  Study. 
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CHAPTER  V.  MODEL  IMPLEMENTATION 

This  chapter  gives  a  suamary  discussion  of  the  cooputer  code 
Inplenentation  of  the  ordinal  rank  order  aggregation  described  in 
Chapter  IV.  The  progranming  computer  code  structure  is  described  and 
the  output  data  format  is  briefly  discussed.  The  computer  code  is 
listed  in  the  appendix.  Extensive  instructions  and  example  computer 
problems  were  documented  in  detail.  However,  because  of  the  large 
volume  of  material,  all  but  the  code  listing  are  published  in  another 
report,  Dobbins  [14]. 

A.  Model  Development  Overview 

The  aggregation  of  multiple  criteria  rank-ordered  priorities  model 
presented  in  Chapter  IV  was  implemented  in  Extended  FORTRAN,  Version  4, 
on  the  CDC  CYBER  74.  The  computer  used  the  NOS/BE  executive  program 
and  has  a  400,000  octal  space  capacity  in  its  central  memory.  The 
computer  facility  is  located  in  the  Scientific  and  Engineering  Division 
of  the  Management  Information  System  Division  of  the  US  Army  Missile 
Connaand  (MICOM) .  Redstone  Arsenal,  Alabama. 

The  code  was  developed  as  an  experimental  program;  therefore, 
achievement  of  its  maximum  matrix  size  was  not  a  major  consideration. 

The  full  potential  for  this  priority  rank-ordered  method  can  be  realized 
when  its  matrix  dimensions  are  re-optimised  for  the  applied  problem  of 
R  &  D  project  prioritization.  Instead  of  the  present  100  x  100  matrix 
dimension,  the  more  practical  dimension  may  be  on  the  order  of  50  x  200 
(50  Judges  with  up  to  200  alternatives). 
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The  code  design  was  isodularized  through  the  use  of  subprograas 
to  facilitate  phased  development*  refinements  during  research*  verifi¬ 
cation*  and  validation.  The  thirteen  subroutine  programs  will  be 
described.  Since  the  design  goal  of  the  model  was  to  form  and  manipu¬ 
late  up  to  100  X  100  element  matrices  and  to  aggregate  one-time  sublists* 
the  batch  processing  mode  of  computation  was  chosen  as  the  most  practical. 

I  B.  Model  Code  Description 

1.  Overall  Computer  Model  Steps 

The  computer  model  for  this  research  requires  large  computer 
core  storage  space  but  operates  very  rapidly  since  It  does  not  use 
Iterative  calculations.  Further,  the  model's  code  design  emphasized 
flexibility  of  programming  options  as  well  as  future  operational 
flexibility  to  input  and  aggregate  a  wide  variety  of  sublist  priority 
order  styles  generated  from  many  ranking  criteria. 

The  flexibility  of  the  model  encompasses  the  wide  variety  of 
sublist  formats  that  have  been  antlclpatcKl*  such  as  requirements  lists* 
expected  operational  dates,  cardinal  data,  and  the  desire  to  develop 
methodology  tools  to  permit  exploration  research  In  such  areas  as  fuzzy 
set  rank  orders,  preference  scoring  constants,  and  comparative  aggrega¬ 
tion  methodologies.  The  comparative  methodologies  Include  the  Borda* 
Adjusted  Borda,  and  the  Shannon  preference  majority-rule  methods, 
a.  Flow  Diagram 

Figure  3  contains  a  simplified  model  flow  diagram. 

A  most  coaiprehenslve  module  Is  the  Input  subroutine.  This  block  of 
the  code  Inputs  and  stores  the  requlrements-to-projects  translation 
equivalency  statements  that  are  expected  to  be  used  for  a  number  of 
runs.  The  Input  also  reads  and  assigns  the  run  and  sublist  control 
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codes  for  actions  such  as  the  matrix  scoring  constants  and  the  weight 
type  for  the  run.  Typical  sublist  controls  are  alternative  identifi¬ 
cations  and  weights.  The  input  also  reads  sublist  data  such  as  the 
ranks  and  self-evaluation  values. 

Found  within  the  input  subprogram,  but  functionally  separate,  is 
the  conversion  of  the  input  sublist  alternatives  to  a  standard  form. 

This  Includes  conversion  of  cardinal  score  alternatives  to  an  ordinal 
ranking  or  the  conversion  of  categorized  alternatives  into  single  rank- 
ordered  lists.  "Alternative"  is  emphasized  at  this  phase  in  the  model 
since  the  ranked  elements  can  be  either  R  &  D  projects  or  product  require¬ 
ments. 

Where  alternatives  are  product  requirements,  the  next  phase  is  to 
translate  those  sublists  of  rank-ordered  requirements  into  transitive 
sublists  of  rank-ordered  R  &  D  projects. 

At  the  completion  of  the  input,  conversion,  and  translation  phases, 
all  sublists  are  ready  to  enter  the  matrix  aggregation  in  a  standard 
form  of  transitive  rank-ordered  lists  of  R  &  D  projects. 

Initially  in  this  second  phase,  the  sublists  become  sublist 
frequency  matrices  scored  by  the  chosen  constants.  The  same  standardized 
sublists  are  used  to  calculate  and  test  the  statistical  significance  of 
the  coefficient  of  concordance. 

If  weighting  and/or  Judge  self-evaluation  are  included  for  the  run, 
the  sublist  matrix  elements  are  next  normalized  by  multiplication  by  4 
and  then  are  weighted.  At  this  phase,  the  Judge  self-evaluation 
factors  become  another  multiplicative  weight. 

The  sublist  matrix  elements,  either  all  weighted  or  unweighted, 
are  sumned  into  the  summed  frequency  matrix  which  contains  the  sum 
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of  votes,  or  weighted  votes,  that  each  alternative  received  when  paired 
against  each  other  alternative.  If  there  Is  no  judge  Indifference, 
the  sum  of  the  values  In  each  row  element  become  the  Borda  count  for 
the  row  (project).  With  or  without  judge  Indifference,  the  sum  of  the 
row  element  values  minus  the  sum  of  the  column  element  values  Is  the 
Adjusted  Borda  count.  The  model  rank  orders  these  counts  Into  the  Borda 
and  Adjusted  Borda  rank  orders. 

The  summed  frequency  matrix  element  values,  divided  by  the  ntimber 
of  judges,  becomes  the  fuzzy  matrix,  R.  From  R,  the  model  calculates 
the  fuzzy  measures,  F(R),  and  C(R),  and  the  fuzzy  rank  order. 

The  comparison  of  the  complement  paired  element  values  In  the 
summed  frequency  matrix  Is  the  basis  for  the  element  values  In  the  pref¬ 
erence  matrix.  The  preference  matrix  assigns  scores  to  projects  for 
the  number  of  majority  comparisons  they  win,  tie,  or  lose.  The  sum  of 
the  row  element  values  provides  the  aggregation  count  for  each  project. 
The  model  rank  orders  this  count  Into  the  aggregation  rank  order. 

The  aggregation  row  counts  also  provide  the  Inputs  for  the  calcu¬ 
lation  of  the  number  of  circular  triads,  D,  and  the  coefficients  of 
consistency,  zeta.  The  model  tests  the  statistical  significance  of  zeta. 

Last,  the  model  can  compare  any  chosen  combinations  of  the  final 
rank  orders  (Borda,  Adjusted  Borda,  Fuzzy,  or  Preference),  then  deter¬ 
mine  and  test  the  Kendall's  Coefficient  of  Concordance  for  these  rank 
orders . 

The  appendix  contains  a  more  comprehensive  model  functional  flow 
diagram  which  contains  major  decision  logic  nodes, 
b.  Subroutine  Programs 

The  appendix  contains  the  listing  and  definitions  of 
key  terms  for  the  aggregation  of  multiple  criteria  rank-ordered 
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priorities  computer  code,  developed  for  the  report  research.  The 
code  structure  diagrasmed  in  Figure  4  consists  of  the  main  program  and 
thirteen  subroutine  program  modules  as  follows: 

1)  Main  Program  -  DOBBINS  -  The  main  program  coordinates 
all  mainstream  processing  of  rank  orders  through  the  model.  It  calls 
subroutines  In  the  proper  sequence  for  calculations  In  a  given  run  based 
on  user  and  model-provided  controls  and  data.  It  writes  only  the  summed 
frequency  matrix  and  the  Borda-type  counts  and  rank  orders. 

2)  Subroutine  INPUT  -  The  subroutine  reads  and  coordi¬ 
nates  the  input  controls  and  data.  The  subroutine  also  converts  the 
sublist  alternative  data  to  the  standard  ordinal  rank-ordered  format. 
INPUT  reads  the  run  controls  and  the  sublist  controls.  It  coordinates 
the  calling  of  subroutine  PRAM  idilch  reads  sublist  ranks,  weights,  and 
self-evaluation  data.  The  self-evaluation  rating  full  scales  are 
converted  to  0  to  1, and  the  specified  sublist  conversions  JCONV  2  to 

12  are  performed  in  INPUT.  This  subroutine  applies  the  self-evaluation 
threshold  and  checks  all  subroutines  for  completion.  Finally  It  stores 
the  converted,  unweighted  standard  form  sublists  of  ranked  alternatives. 

3)  Subroutine  PRAM  -  This  is  a  library  subroutine  to 
enter  floating  point  data  in  free  format  form  where  precise  formats 
are  not  practical.  In  this  computer  model,  PRAM  is  used  to  enter  the 
siibllst  rank  data  and  the  self-evaluation  data. 

4)  Subroutine  REQUIR  -  This  subroutine  receives 
converted  requirements  sublists  from  INPUT.  It  compares  each  sublist 
Co  the  cranslatj..;i  Index  that  has  been  sorted  and  arranged  by  requlre- 
■nC  aame.  RE()UIR  then  extracts  the  projects  that  match  the  requirement 
la  Che  awbllsc.  A  project's  rank  order  is  built  by  insertion  of  the 

ef  prejecCs  for  each  requirement.  REQUIR  purges  duplications 
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from  the  raw  projects  rank  order.  The  transitive  project  subllst  Is 
returned  to  INPUT. 

5)  Subroutine  FREQ  -  This  subroutine  coordinates  the 
placement  of  the  subllsts  Into  subllst  frequency  matrices  and  the 
weighting  of  the  frequency  matrix  elements.  It  further  applies  the 
self-evaluation  values  to  the  frequency  matrix  elements  and  writes  the 
subllst  self-evaluation  frequency  matrix. 

6)  Subroutine  MATRIX  -  This  subroutine  forms  and  writes 
the  subllst  frequency  matrix  for  each  subllst.  The  matrices  are  formed 
using  the  specified  matrix  scoring  constants. 

7)  Subroutine  WEIGHT  -  This  subroutine  applies  the 
specified  weighting  to  each  subllst  frequency  matrix  element  and  writes 
the  weighted  subllst  frequency  matrix.  Before  any  weights  are  applied, 
HEIQIT  multiplies  all  subllst  matrix  elements  by  four. 

8)  Subroutine  PREF  -  This  subroutine  forms  and  writes 
the  preference  matrix,  calculates  the  nuiid>er  of  circular  triads,  D,  and 
the  coefficient  of  consistency,  zeta,  and  statistically  tests  zeta.  The 
subroutine  also  calculates  the  bracket  and  average  values  for  D  and 
zeta  when  fractional  sums  occur  In  the  preference  matrix  rows.  All  D, 
zeta,  and  test  results  are  output  by  this  subroutine.  The  matrix  Is 
formed  using  the  specified  matrix  scoring  constants. 

9)  Subroutine  MUCH  -  This  subroutine  Is  an  International 
Mathematical  and  Statistical  Libraries  (IMSL)  program  which  Is  used  for 
the  chl-squared  probability  statistical  tests  of  the  Kendall  coefficient 
of  concordance  and  the  coefficient  of  consistency.  MUCH  automatically 
changes  to  use  the  normal  distribution  approximation,  Z,  for  the  high 
degree  of  freedom  with  the  chi-square  statistic  (dF  ^  30) . 
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10)  Subroutine  ORDER  -  This  subroutine  converts  a  list 

of  values  for  a  set  of  alternatives  Into  an  ordinal  rank-ordered  list  of 
the  alternative  Identifications.  ORDER  writes  the  final  raidc-ordered 
list.  ^ 

11)  Subroutine  FUZZY  -  This  subroutine  calculates  the 
fuzzy  matrix,  the  measures  F(R)  and  C(R),  and  writes  the  Fuzzy  project 
scores.  This  subroutine  coordinates  the  forming  and  writing  of  the 
Fuzzy  rank-ordered  list. 

12)  Subroutine  CONCOR  -  This  subroutine  calculates, 
statistically  tests,  and  outputs  the  results  of  the  coefficient  of 
concordance  for  the  standard  subllsts  and  the  final  results.  CONCOR 
also  calculates  and  outputs  the  Intermediate  concordance  variables 
such  as  the  mean  and  the  sum  of  the  squares  of  the  deviations  from  the 
mean. 

13)  Subroutine  CCMPARE  -  This  subroutine  takes  the 
final  aggregate  rank  orders  from  the  BORDA,  ADJ.  BORDA,  PREFERENCE, 
and.  If  available,  FUZZY  methods  and  compares  them  two  at  a  time.  The 
pairs  of  aggregated  lists  are  sent  to  CONCOR  and  evaluated,  then  control 
returns  to  COMPARE.  When  the  last  pair  Is  evaluated,  COMPARE  returns  to 
the  main,  DOBBINS,  and  the  model  terminates. 

2.  Input  to  the  Computer  Model 
a.  Overview 

The  Inputs  to  this  computer  code  have  been  kept  relatively 
simple  compared  to  the  complexity  of  the  model.  Inputs  are  the  run 
controls,  the  alternative  names,  the  ranking  and  rating  data  which  are 
entered  In  free  format. 


b.  Option  Control 


Besides  the  alternative  names  and  niJ]id)ers,  the  first  set 
of  cards  (one  card  per  alternative)  contain  the  alternative  weight  factors 
and  category  for  that  alternative.  The  second  type  of  single  control 
card  has  Integer  numerical  digits  to  designate  the  type  weighting  technique 
(1  through  8),  whether  the  sublists  should  be  completed*  the  type  of 
matrix  scoring  constants  for  both  the  frequency  and  preference  matrices* 
the  self-evaluation  control  for  weighting  or  weighting  and  threshold 
elimination*  and  the  self-evaluation  scale  limit  value.  The  third  type 
control  card  (one  card  per  Judge  sublist)  Identifies  the  Judge*  designates 
the  alternative  conversion  type*  and  gives  the  Judge  weight  factors.  The 
appendix  specifies  Input  data  format  In  greater  detail, 
c .  Data 

The  sublist  ranks  and  the  self-evaluation  ratings  are 
Input  as  separate  sets  of  free  format  cards.  For  the  sublist  rank  set* 
the  sequence  of  alternative  numbers  Indicates  the  preference  order  with 
minus  signs  used  to  Indicate  equality  or  indifference.  Each  sublist 
ends  with  an  asterisk.  The  sublist  self-evaluation  ratings  are  listed 
on  their  cards  In  an  order  corresponding  to  the  lexlgraphic  order  of 
the  alternatives'  Identification  numbers  (1*  2*  3*  ...etc.). 

3.  Output  from  the  Computer  Model 
a.  Aggregation  Rank  Orders 

The  primary  output  of  the  computer  model  for  this  research 
Is  the  aggregated  rank-ordered  list  of  R  &  D  projects.  The  Shannon 
majority  method  produces  the  baseline  aggregation  rank-ordered  list  for 
the  model.  For  comparative  purposes*  the  model  also  produces  the  Borda- 


outputs  the  inputs,  the  sublist  aatrlces  (basic,  and  if  appropriate, 
the  weighted  and/or  self-evaluation  matrices)  and  the  sum  of  the  rows 
for  the  sunmed,  Fhzzy,  and  preference  matrices, 
b.  Evaluation  Results 

The  model  further  provides  the  results  of  the  evaluation 
of  the  input  sublists  and  the  aggregation  rank  orders.  It  computes, 
ststlstlcally  tests,  and  prints  the  major  steps  of  the  Kendall  coeffi¬ 
cient  of  concordance ' evaluation  of  the  standardized  input  sub lists.  The 
statistical  tests  conclude  with  statements  as  to  whether  the  input  rank 
orders  are  consistent  at  the  0.05  and  0.01  significance  levels.  Again, 
for  verification  and  analysis  of  each  evaluation,  the  model  provides  the 
rank  array,  the  alternative  sums,  means,  the  sum  of  the  squared  deviations, 
tied  ranking  factor,  and  the  coefficient  of  concordance. 

The  model  also  performs  up  to  six  Kendall's  coefficient  of  concor¬ 
dance  analyses  of  all  two-rank-order  combinations  of  the  four  aggregation 
rank  orders,  i.e..  Shannon  versus  Fuzzy,  Shannon  versus  Borda,  Adjusted 
Borda  versus  Fuzzy,  etc.  The  output  details  are  the  same  as  those  for 
the  coefficient  of  concordance  evaluation  for  the  input  sublists.  These 
evaluation  data  provide  a  measure  of  the  agreement  between  the  various 
final  aggregations. 

The  other  evaluation  paraawters  are  Kendall's  number  of  circular 
triads,  D,  and  the  coefficient  of  consistency,  zeta,  which  evaluate 
the  cyclicity  characteristics  of  the  Shannon  aggregation  rank  order. 

The  statistical  tests  determine  if  the  tested  rank  order  could  have 
occurred  by  chance.  Instead  of  by  a  somewhat  consistent  preference  method. 

The  appendix  contains  the  program  list  for  the  caaq>uter  model. 


C.  Hacrlx  and  Rank  Order  Formation 

Once  the  aubllat  rank  ordera  are  In  standardized  project  alter¬ 
native  formatSt  the  computer  model  forms  each  into  a  sublist  frequency 
matrix  that  Indicates  which  project  is  preferred  over  each  other  project 
by  pair  comparisons.  The  summed  frequency  matrix  is  the  matrix  element 
addition  of  the  sublist  frequency  matrices.  The  preference  matrix  is 
formed  from  the  paired  comparisons  of  each  of  the  summed  frequency 
matrix  element  values. 

The  project  scores  are  computed  from  the  row  and  column,  if  appro¬ 
priate,  sums  of  the  elements  of  the  summed  frequency  matrix  and  the 
preference  matrix.  The  model  then  places  the  project  with  the  highest 
score  highest  In  the  rank  order  and  repeats  the  search  for  each  equal 
or  lower  scored  project. 

D.  Ancillary  Processes 

1.  Weighting 

a.  Decislon-Haker  Hethods 

The  model  Input  weighting  functions  are  Input  as  a  control 
code  and  weighting  factor  data  codes.  The  weighting  factors  are  the 
weights  applied  to  each  alternative,  WHI,  and  the  weights  applied  to 
each  Judge,  WHJ.  If  Alternative  a  has  a  WHI  value  other  than  1,  every 
time  Alternative  a  appears  in  a  sublist  frequency  matrix.  It  will  be 
weighted  by  the  factor  WHI.  If  Judge  2  has  a  WHJ  value  other  than  1, 
every  alternative  In  Judge  2*8  sublist  will  be  weighted  by  the  factor 
WHJ.  If  Alternative  a  Is  In  Judge  2's  sublist,  it  will  be  dual-wei^ted 
by  WHI  and  WHJ. 

b.  Judge  Self-Evaluation  Hethods 

The  Judge  self-evaluation  (JSE)  methodology  is  imple¬ 
mented  as  a  weighting  scheme.  JSE  Is  controlled  by  the  HAIR  code  in 
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Che  control  card,  as  follows: 

HAIR  -  0  -  No  JSE. 

MATR  >  1  -  The  JSE  factors  are  applied  to  all  raidied  alternatives. 
No  threshold  Is  applied. 

MATE  -  2  -  The  JSE  factors  are  applied  to  all  ranked  alternatives. 
A  threshold  Is  applied  that  purges  all  ranked  alternatives  with  JSE 
ratings  below  the  threshold  value  which  Is  Input  as  a  THLD  value. 

2.  Evaluation  Techniques 

No  controls  are  necessary  to  obtain  evaluation  of  the  Input 
rank  orders.  The  final  aggregation  rank  orders  are  comparatively 
evaluated  by  the  coefficient  of  concordance  method  for  each  combination 
of  final  rank  orders  that  Is  calculated  by  the  model.  The  evaluation 
and  test  calculation  techniques  are  described  In  Chapter  IV  and  Dobbins 
fl41. 
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CBAPTER  VI.  HOm  VERIFICATION  AMD  VALIDAIKW 


This  chapter  contains  the  verification  and  validation  of  the 
coaputer  model,  including  sample  numerical  validation  problems. 

A.  Verification 

1.  Model  Design  and  Test 

Verification  that  the  rank-ordered  aggregation  computer  model 
was  implemented  properly  in  the  computer  code  was  accomplished  through 
the  modular  design  of  the  code,  unit  testing  of  each  subroutine,  phased 
buildup  of  the  computer  model  with  tests  after  each  phase  was  added, 
running  a  series  of  test  problems  for  comparisons  of  computer  model 
output  with  hand-calculated  results,  and  a  final  exercise  of  all  options 
in  the  program. 

Extensive  model  validation  in  the  sense  of  running  large  aggrega¬ 
tion  rank-ordered  priorities  problems  was  not  possible  due  to  the  lack 
of  available  problems  with  known  solutions  using  any  of  the  four 
majority-rank  methods  that  are  built  into  the  computer  model.  Valida¬ 
tion  of  portions  of  the  model  options  against  moderate-slxed  known 
problems  with  solutions  from  the  literature  was  accomplished.  Some  of 
the  special  features  of  this  model,  such  as  weighting,  fussy  rank  orders, 
and  Judge  self-evaluation  were  validated  by  calculated  extensions  from 
matrix  aggregation  methods  confirmed  against  the  literature. 

2.  Verification  Dcsionstratlon 

The  computer  model's  flexibility  was  verified  and  demonstrated 
in  Dobbins  (14)  through  the  exercise  of  most  of  the  computation  options 


for  a  single  set  of  partial  aubllst  rank  orders*  a  set  of  alternative 
and  Judge  weights*  and  a  set  of  self-evaluation  ratings. 

B.  Model  Validation 

Computer  model  validation  was  accomplished  by  comparing  results 
of  the  computer  model  to  results  for  examples  found  in  the  literature. 

The  literature  often  gave  only  winners  for  the  method  employed.  The 
validation  was  divided  into  areas  of  method  anphasis  in  the  literature 
examples  as  follows:  Borda  and  Adjusted  Borda;  Borda,  condorcet,  and 
Black;  Borda,  Condorcet,  Black  and  Copeland;  Copeland;  Shannon  pref¬ 
erence  and  others;  and  special  purpose  examples  to  validate  other  model 
areas  such  as  tied  data  and  evaluation  tests.  Each  case  in  Tables  4 
through  9  presents  selected  literature  examples,  published  results, 
comparable  results  from  the  computer  model,  and  additional  model  results. 
A  total  of  46  literature  cases  was  validated,  but  only  13  are  in  this 
report.  All  46  validation  cases  are  published  in  Dobbins  (14). 

All  six  tables  have  the  same  format.  The  left  one  half  of  each 
table  page  is  quoted  from  the  literature.  First,  the  reference 
identification  is  listed;  then  the  example  sublist  rank  orders  are  shown. 
Last*  key  answers  from  the  literature  are  given.  The  right  one  half 
of  each  table  page  contains  results  from  aggregating  the  literature 
example  sublists  in  the  computer  model.  The  upper  left  portion  of  the 
computer  model  side  of  the  page  contains  the  various  final  rank  orders 
as  computed.  The  upper  right  portion  of  the  model  side  of  the  page 
contains  the  results  of  the  coefficient  of  consistency  testing  of  the 
preference  matrix.  The  lower  portion  of  the  model  side  of  the  page 
contains  coefficient  of  concordance  results  for  the  sublists  and  for 
selected  pairs  of  final  aggregation  results. 
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For  Case  1  of  Table  4»  Rlchelson  preaented  X  as  the  Borda  winner 

% 

which  was  in  agreement  with  both  computed  Borda  orders.  In  Case  2,  the 
)  literature  example  gave  the  Borda  and  the  Adjusted  Borda  counts.  For 

Case  2,  where  there  were  no  ties  in  the  sublist  ranks*  the  computer 
data  fully  agreed  with  the  literature  examples. 

For  Case  2,  the  asterisk  (*)  at  the  sublist  concordance  results 
denotes  that  the  examples  Indicated  contained  repetitions  of  sublists. 

For  these  cases,  the  repetitions  of  sublists  were  input  as  multiplicative 
Judge  weights  <Wj) .  The  rank  orders  were  the  same  but  the  sublist 
concordance  data  were  based  on  single  occurrences  of  each  type  of  sublist. 
It  was  concluded  that  the  model  adequately  represented  the  Borda  and 
Adjusted  Borda  majority-rank  methods. 

For  the  cases  of  Table  5*  for  Borda*  Condorcet*  and  Black  method 
examples*  the  Borda  results  were  computed*  and  the  Condorcet  results 
were  observed  by  scanning  the  rows  of  the  preference  matrix  for  zeros. 

If  a  zero  (other  than  on  the  main  diagonal)  existed*  then  the  alternative 
did  not  have  a  majority  over  all  other  alternatives*  which  is  the 
Condorcet  criterion.  The  Black  answer  is  the  Condorcet  winner  if  one 
exists.  If  a  Condorcet  winner  did  not  exist*  the  Black  winner  is  the 
Borda  or  Adjusted  Borda  winner.  A  strong  Condorcet  winner  is  one  that 
beats*  not  ties*  all  other  alternatives.  For  Table  5*  computed  results 
for  both  cases  agreed  with  the  literature  examples.  For  case  2*  the 
Borda  count  values  were  also  given  in  the  literature  and  were  in  agree¬ 
ment  with  the  computer  model  results. 
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MODEL  VALIDATION  WITH  LITERATURE 


IZ  Cone 


TABLE  4.  (CONCLUDED) 
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TABLE  5.  (CONCLUIXD) 


Cocff  W:  0.190  0.875 


For  tho  cases  of  Table  6»  tbs  Opelaad  results  were  obtained  as 

the  preference  order  of  the  model  when  1,  l/2»  0  scoring  constants 

were  used  for  the  frequency  matrices  and  1»  0,  -1  scoring  constsmts  were 

i 

used  for  the  preference  matrix.  The  other  results  ware  obtained  as  In 
Table  5.  Cases  1  and  2  of  Table  6  have  total  corre^ondence  between 
literature  and  compute:  results  from  the  rank>K>rdered  aggr^atlons. 

The  aK>del  Is  validated  for  the  Copeland  method.  Both  cases  in  Table  7 
have  Copeland  results  from  the  literature  and  the  model  that  fully  agree. 

case  1  of  Table  8  has  full  agreement  between  the  literature 
examples  and  the  model  results.  In  Case  2,  the  rank  orders  agreed  but 
the  sublist  concordance  figures  differed  because  of  an  error  in  the  sum 
of  the  ranks  for  the  F  alternative.  Dr.  Shannon,  author  of  Case  2,  told 
of  the  error  during  a  class  lecture.  Both  cases  In  Table  8  validated 
the  Shannon  preference  method  in  the  model. 

Table  9  presents  special  cases  to  validate  specific  functions  in 
the  model.  Case  1  Is  a  Kendall  example  to  illustrate  the  ties 
correction  calculation  of  the  coefficient  of  concordance  when  signif¬ 
icant  ties  (indifferences)  were  In  the  sublists.  The  literature  example 
and  the  model  computation  of  Case  1  agreed  completely. 

Case  2  Is  a  Kendall  example  to  Illustrate  the  number  of  circular 
triads  (D)  and  coefficient  of  consistency  (seta)  computations.  The 
literature  example  began  with  the  preference  matrix  and  continued  to 
the  completion  of  the  consistency  evaluation.  The  literature  example 
and  the  model  computation  of  Case  2  agreed. 


TABLE  6.  MODEL  VALIDATKM  WITH  LITERATUEE  -  III 


5Z  Cone 


TABLE  6.  (CONCLUDED) 


TABLE  7.  MODEL  VALIDATION  WITH  LITERATURE 


(CONCLUDED) 


TABLE  8.  (QONCLUKD) 


5X  Cone: I  Yes  I  Yes 


TABLE  9.  MODEL  VALIDATION  WITH  LITERATUBE 


Co«ff  W;  0.828  0.994 


TABLE  9.  (CONCLUDED) 


Ca««  3  is  snothsr  Kendall  cocample  to  Illustrate  the  ealculatioii  of 
the  Kendall's  coefficient  of  concordance,  W.  The  literature  exaaple 
and  the  nodal  calculation  of  the  nean,  the  square  of  the  deviation,  and 
the  coefficient  of  concordance  all  agreed. 

In  suasMry,  the  literature  cases  and  node!  results  In  the  sis 
tables  represent  a  reasonable  validation  of  the  model. 


CHAPTER  VII.  R  &  D  PROJECTS  PRIORITIZATION  STUDY 


This  chapter  contains  a  case  study  of  the  application  of  the 
ordinal  rank  order  aggregation  method  and  model.  The  problem  Is  that 
of  establishing  the  prioritization  of  R  &  D  projects. 

A._  Overview 

The  objective  of  this  research  has  been  to  develop  a  methodology 
to  aggregate  multiple  sourced  rank-ordered  lists  of  product  requirements 
and  R  &  D  projects  Into  a  single  list  of  rank-ordered  R  &  D  projects. 

In  this  chapter,  the  methodology  toward  this  objective  will  be  demon¬ 
strated.  The  methodology  will  employ  the  aggregation  computer  model 
described  in  previous  chapters.  The  Shannon  majority-rule  preference 
method  will  be  used  for  the  final  rank-ordered  list  of  projects.  The 
input  sublists  of  ordinal  product  requirements  and  R  &  D  projects  will 
be  used  and  documented  In  this  chapter.  The  demonstration  study  will 
aggregate  at  least  thirteen  sublists  for  95  R  &  D  projects 
(alternatives) .  The  requirements  sublists  will  rank  44  product  require¬ 
ments.  The  sublist  frequency  matrix  and  weighted  sublist  matrix  each 
require  seven  computer  printed  pages  for  the  95  x  95  matrices.  There¬ 
fore,  the  approximately  182-page  total  computation  printing  for  the 
sublist  matrices  will  not  be  Included  in  this  report.  Complete 
problem  computation  examples  can  be  found  In  Dobbins  (14)  for  a 
smaller  dimensioned  problem.  Sample  sublist  data  and  sumnary  results 
will  be  Included  for  the  R  &  D  project  demonstration  study. 
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To  avoid  any  national  aecurlty  concerns*  the  naaiea  and  the 
correct  Identification  numbers  of  the  Individual  product  requlroients 
and  R  &  D  projects  will  not  be  used.  A  typical*  fictitious  product 
requirement  would  be  R601*  to  obtain  a  new  hand-held*  10-km  range 
antipersonnel*  all  weather  rocket^propelled  weapon  system.  A  typical* 
fictitious  R  &  D  project  would  be:  P6D-65*  to  develop  the  technology 
to  locate*  Identify*  and  guide  a  missile  to  the  personnel  targets  at 
night  In  rain,  fog,  and  smoke.  It  could  be  determined  that  PGD-65  was 
the  only  R  &  D  project  necessary  to  develop  product  requirement  R601. 

If  that  were  the  case,  then 

R601.  H  PGD-65. 

would  be  the  translation  Index  for  R601  and  PGD-65.  The  translation 
Index  permits  translation  of  requirements  rank-ordered  lists  Into 
R  &  0  project  rank-ordered  lists. 

B.  computation  Inputs 

To  perform  the  R  &  D  project  prioritization  demonstration*  several 
subllsts  of  judge  rank  orders  must  be  Input  Into  the  model.  The 
characteristics  of  the  laboratory  management  envlronnent  and  the  subllsts 
will  be  described  as  each  Is  given.  The  necessary  controls  for  study 
and  for  the  translations  from  requirements  will  be  described.  Last* 
the  weighting  technique  and  values  will  be  discussed. 

1.  Subllsts  Data 

The  R  &  D  laboratory  Is  faced  with  multiple  sources  of 
suggested  rank  ordering  of  Its  R  &  D  projects.  These  sources  include 
requirements  studies  and  other  documents*  headquarters  management*  and 
the  local  laboratory  laanagement.  The  laboratory  director*  as  the 
formal  decision-maker*  must  evaluate  each  sublist  source  and  each 


alternative  to  detemlne  what  special  eaphasls  should  be  given  to 
each.  The  alternatives  are  considered  of  equal  special  anphasls  for 
purposes  of  this  prioritisation  study.  Therefore,  each  alternative  is 
given  an  equal  alternative  weight  of  one.  The  Judge  sublists  are 
given  judge  weights  corresponding  to  their  sources.  For  exaaple,  each 
directorate  manager's  sublist  has  a  weight  of  five,  while  the  laboratory 
director's  subllst  has  a  weight  of  ten. 

a)  R  &  D  Projects  Sublists 

The  laboratory  director  has  his  own  preferences  for  the 
rank-ordered  priorities  of  this  R  &  D  projects.  In  an  autocractlc 
organisation,  that  list  would  prevail  and  this  study  would  be  a  mute 
effort.  But  In  a  partlclpatlvely  managed  organisation,  the  director 
will  choose  to  give  reasonable  consideration  to  the  recommendations  of 
his  Inferior  managers  as  wrell  as  those  of  his  superiors. 

The  alternative  R  &  D  projects  have  been  numbered  with  index  values 
from  1  to  95.  Each  alternative  project  has  a  weight  of  one.  In 
addition,  each  project  is  identified  as  a  member  of  a  technology 
category.  In  the  prior  example  for  PGD-65,  the  GD  symbols  represent 
the  technology  category  for  this  project.  The  laboratory  director 
concentrates  his  attention  on  the  management,  balance,  and  resources 
of  the  technology  categories,  while  the  directorate  managers  concentrate 
on  managing  the  projects  within  a  category.  Therefore,  the  laboratory 
director's  sublist  will  be  a  rank-ordered  list  of  the  technology 
categories.  The  model  will  use  this  ranked  category  list,  with  Its 
corresponding  projects,  to  develop  a  single  list  of  projects  for  the 
director.  The  study  will  Include  the  option  case  where  he  considers 
the  projects  as  unranked  within  each  category,  and  the  option  case 
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vh«re  he  accepts  the  dlxeetorate  ■aoagers  recfaneodations  for  the  project 
rankiogs  within  each  category.  The  laboratory  director's  technology 
category  preferences  are  listed  in  Table  10. 
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TABLE  10.  LABORATORY  DIRECTOR'S  SUBLIST 


Rank 

Category 

Index 

Category 

Indentlflcatlon 

1 

1 

ES 

2 

2 

GG 

3 

3 

ET 

4 

5 

GD 

5 

6 

DS 

6 

7 

KP 

7 

4 

EHG 

Each  directorate  manager  ranked  the  R  &  D  projects  within  his 
category.  The  sublists  for  each  category  manager  are  listed  in  Table  11. 


TABLE  11.  CATEGORY  MANAGER'S  SUBLIST 


Category  ES  | 

Rank 

Project 

Index 

Project 

Indentlflcatlon 

1 

1 

PES-1 

2 

2 

PES-2 

3 

3 

PES-3 

4 

4 

PES-4 

5 

5 

PES-5 

6 

6 

PES-6 

7 

7 

PES-7 

8 

8 

PES-8 

9 

9 

PES-9 

10 

10 

PES-10 

11 

11 

PES-11 

12 

12 

PES-12 

13 

13 

PES-13 

14 

14 

PES-14 

15 

15 

PES-15 

16 

16 

PES-16 

table  11.  (CONTINDED) 


1  Category  GG  | 

Project 

Project 

Project  . 

Project 

Rank 

Index 

Indent  if  lea  t  ion 

Rank 

Index 

Indentification 

17 

PGG-17 

20 

36 

PGG-36 

18 

PGG-18 

21 

37 

PGG-37 

19 

PGG-19 

22 

38 

PGG-38 

20 

PGG-20 

23 

39 

PGG-39 

5 

21 

PGG-21 

24 

40 

PGG-40 

6 

22 

PGG-22 

25 

41 

PGG-41 

7 

23 

PGG-23 

26 

42 

PGG-42 

8 

24 

PGG-24 

27 

43 

PGG-43 

9 

25 

PGG-25 

28 

44 

PGG-44 

EH 

26 

PGG-26 

11 

27 

PGG-27 

.12 

28 

PGC-28 

13 

29 

PGG-29 

14 

30 

PGG-30 

15 

31 

PGG-31 

16 

32 

PGG-32 

17 

33 

PGG-33 

18 

34 

PGG-34 

19 

35 

PGG-35 

Category  ET 

Categor; 

r  GD 

Project 

Project 

Project 

Project 

Rank 

Index 

Indent if Icat ion 

Rank 

Index 

Indentification 

45 

PET-45 

46 

PET-46 

58 

PGD-58 

47 

pet-47 

59 

PGD-59 

48 

pet-48 

60 

PGI7-60 

49 

pet-49 

61 

PGD-61 

50 

PET-50 

5 

62 

PGD-62 

51 

PET-51 

6 

63 

PGD-63 

8 

52 

PET-52 

7 

64 

PGD-64 

kh 

53 

PET-53 

8 

65 

PGD-65 

54 

PET-54 

66 

PGD-66 

Catei&ury  EHG  I 

M 

67 

PGD-67 

1 

55 

PEHG-S5 

69 

PGD-69 

2 

56 

PEHG-56 

70 

PGD-70 

3 

57 

PEHG-57 

71 

PGD-71 

1 


I 
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TABLE  11.  (CONCLUDED) 


Category  DS  | 

1  Category  KP 

Rank 

Project 

Index 

Project 

Indentlflcation 

Rank 

Project 

Index 

Project 

Indent If Icatlon 

1 

72 

PDS-72 

83 

PrP-83 

2 

73 

PDS-73 

84 

PKP-84 

3 

74 

PDS-74 

85 

PKP-85 

4 

75 

PDS-75 

86 

PKP-86 

76 

PDS-76 

5 

87 

PKP-87 

7.7 

PDS-77 

6 

88 

PKP-88 

78 

PDS-78 

7 

89 

PKP-89 

8 

79 

PBS-79 

8 

90 

PKP-90 

80 

PDS-80 

mSm 

91 

PKP-91 

81 

PDS-81 

92 

PKP-92 

82 

PDS-82 

93 

PKP-93 

94 

PKP-94 

IHI 

■IM 

95 

PKP-95 

The  laboratory  director's  staff,  fron  its  nore  analytical  viewpoint 
than  the  director's,  ranks  the  projects*  The  top  thirty  Items  sublist 
for  the  laboratory  staff  are  listed  In  Table  12. 

TABLE  12.  laboratory  STAFF  SDBLIST 


Rank 

Project 

Index 

Rank 

Project 

Index 

1 

1 

16 

95 

2 

3 

17 

62 

3 

4 

18 

50 

4 

2 

19 

47 

5 

7 

20 

49 

6 

45 

21 

44 

7 

59 

22 

32 

8 

72 

23 

27 

9 

80 

24 

28 

78 

25 

24 

79 

26 

22 

83 

27 

15 

89 

28 

10 

92 

29 

6 

94 

30 

11 

111 


Several  trl-eervice  coMiittees  cxiet  which  study  restricted  sress 


of  DOD  technology  sctlvitles  sad  nsks  recoanendstioiis  to  improve  the 
effectiveness  sod  efficiency  of  technology  sdvsncsment  efforts  of  the 
DOD  Isborstories.  These  reconmendstlons  also  identify  sod  remedy 
overlap  and  balance  between  laboratories.  In  this  demonstration  study, 
the  laboratory  director  has  been  provided  a  recommended  priority  list 
for  his  laboratory's  projects  by  the  DOD  tri-service  Terminal  Guided 
Subaunltlons  (TGSH)  Coenlttee  and  the  DOD  Antitank  Guided  Munitions 
(AIGM)  Coaalttee.  The  sublists  for  the  TGSM  Conmlttee  and  the  ATGM 
Committee  are  listed  in  Table  13. 

TABLE  13.  TGSM  COMMITTEE  SDBLIST 


Rank 

Project 

Index 

Rank 

Project 

Index 

Rank 

Project 

Index 

1 

2 

7 

4 

13 

2 

45 

8 

5 

14 

3 

49 

9 

10 

15 

4 

51 

10 

12 

16 

5 

46 

11 

11 

17 

14 

6 

47 

12 

15 

ATGM  COtOaTTEE  SUBLIST 


Project 

Index 

Rank 

Project 

Index 

2 

7 

28 

1 

8 

12 

7 

9 

83 

47 

87 

17 

11 

95 

27 

12 

93 

Rank 


The  laboratory  director,  as  the  declslon-aaker,  determines  the 
weights  to  be  applied  to  each  Judge's  subllst.  The  weights  will  be 
applied  by  weighting  type  two  tdtlch  Is  nultlpllcatlve.  The  weights 
for  the  Judges  who  prepared  R  &  D  project  subllsts  are  In  Table  14. 

TABLE  14.  projects  JUDGE'S  WEIGHTS 


Judge 

Height 

(WTJ) 

Laboratory  Director 

10 

Category  ES  Manager 

3 

Category  GG  Manager 

3 

Category  ET  Manager 

3 

Category  EHG  Manager 

3 

Category  GD  Manager 

3 

Category  Manager 

3 

Category  KP  Manager 

3 

Laboratory  Staff 

.  5 

TGSM  Committee 

5 

ATGH  Comnittee 

5 

2.  Requirements  Subllsts 

Two  requirements  priority  lists  are  available  for  the 
decision  maker  to  use  In  this  R  &  D  projects  prioritisation  study.  The 
alternative  product  requirements  have  been  numbered  with  Index  values 
from  1  to  44.  Each  alternative  requirement  has  a  weight  of  one.  In 
addition,  each  requirement  Is  Identified  as  a  number  of  a  requirement 


category. 


One  requlzementa  subllet*  BOP-ST  COM,  rank  ordera  all  of  tbe 
requircaienta  baaed  on  one  categorization  of  ultinate  utilization.  The 
aubllat  for  BDP-ST  COM  la  Hated  In  Table  15. 


TABLE  15.  BDP-ST  COM  REQUIBBMENTS  SUBLIST 


Rank 

Reqmt . 
Index 

Reqnt. 

Ident. 

Category 

Rank 

Reqnt. 

Index 

Reqnt. 

Ident. 

Category 

■■ 

1 

RlOl. 

23 

23 

R401.00I 

4 

2 

R106 

1 

24 

24 

R401.00K 

4 

3 

R108. 

1 

25 

25 

R401.00N 

4 

4 

R204. 

2 

26 

26 

R402 

4 

5 

5 

R213. 

2 

27 

27 

R404 

4 

6 

6 

R301. 

3 

28 

28 

R405 

4 

7 

7 

R302.00A2 

3 

29 

29 

R501.7 

5 

8 

8 

R302.00E2 

3 

30 

30 

R502.2 

5 

9 

9 

R303 

3 

31 

31 

R601 

6 

10 

10 

R304 

3 

32 

32 

R602. 

6 

11 

11 

R305.00F 

3 

33 

33 

R603 

6 

12 

12 

R306 

3 

34 

34 

R604 

6 

13 

13 

R307 

3 

35 

35 

R606 

6 

14 

14 

R309 

3 

36 

36 

R607 

6 

15 

15 

R310.00I 

3 

37 

37 

R609 

6 

16 

16 

R310.00K 

3 

38 

38 

R610 

6 

17 

17 

R312.00B 

3 

39 

39 

R611 

6 

18 

18 

R313.00A 

3 

40 

40 

R612 

6 

19 

19 

R401.00A 

4 

41 

41 

R615 

6 

20 

20 

R401.0QB 

4 

42 

42 

R616 

6 

21 

21 

R401.00G 

4 

43 

43 

R701.8 

7 

22 

22 

R401.00H 

4 

44 

44 

R802 

8 

In  the  prior  cxaaple  for  R601,  the  60  Integere  represent  the 
requirements  category.  In  the  requirements  subllst,  ST>  which  uses 

I 

categories*  the  requirements  categories  (1  through  8)  are  unranked  and 
considered  equal.  The  requirements  within  each  category  are  ranked. 
The  subllsts  for  each  requirements  category  are  listed  In  Table  16. 
TABLE  16.  SEQUIBBMBNTS  CATEGORY  SUBLISTS 


Category  STl 

Category  ST2 

Rank 

Project 

Index 

Rank 

Project 

Index 

1 

1 

1 

4 

2 

2 

2 

5 

3 

3 

Category  ST3 

Category  ST4 

Rank 

Project 

Index 

Rank 

Project 

Index 

TABLE  16.  (CGNCLDDED) 


Category  STS 

Catei 

{ory  ST6 

Rank 

Project 

Index 

Rank 

Project 

Index 

1 

29 

1 

31 

2 

30 

2 

32 

3 

33 

4 

34 

5 

35 

6 

36 

7 

37 

8 

38 

9 

39 

40 

11 

41 

IZ 

42 

Category  ST7 

Category  STS 

Rank 

Rank 

Project 

Index 

Rank 

Project 

Index 

1 

43 

1 

44 

The  decision-maker  also  assigned  weight  for  the  requirements 
Judges.  The  weights  are  given  In  Table  17. 

TABLE  17.  BEQUIREMENTS  JUDGE'S  WEIC»TS 

I  . .  I 


Sublist 


Weight  (WHT) 


2.  Computation  Option  Controls 
a)  Options 

*  • 

1)  Weights  -  The  multiplicative  weighting  method 

(MWI  -  2)  was  chosen  for  this  study  with  the  alternative  weights  equal 
to  one  and  the  judges'  weights  Input  as  integers.  Though  the  other 
methods  were  considered »  the  multiplicative  was  chosen  for  convenience 
to  the  decision-maker. 

2)  Completion  -  Most  sublist  inputs  for  this  study 
are  Incomplete  since  most  judges  chose  to  rank  the  projects  or  require¬ 
ments  which  related  to  their  areas  Of  specialization.  The  study,  using 
JCONV  3,  will  consider  an  alternative  case  where  all  sublists  are 
completed  by  adding  all  unranked  alternatives  as  equal  and  lower  rank 
than  the  list.  This  will  give  a  comparison  with  the  uncompleted  case 
to  determine  If  any  alternatives  are  moved  to  other  sections  of  the 
final  priority  list. 

3)  conversions  -  The  data  for  this  study  required 
conversion  for  two  ^blists.  The  laboratory  director's  sublist  will 
be  one  of  ranked  R  &  D  project  categories.  The  study  will  examine 
both  the  case  where  the  director  considered  the  projects  equal  in  each 

category  and  the  case  where  the  director  accepted  the  rankings  of  each 

a 

category  manager. 

The  second  conversion  will  be  that  for  requirements  where  the 
categories  are  unranked,  but  the  requirements  within  each  category  are 

ranked.  In  each  case  above,  the  model  will  convert  the  categories  and 

a 

their  alternatives  Into  single  prioritized  sublists. 
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b)  Translation  Equivalences 

1)  Methodology  -  Translation  of  the  ranked  requlreaents 
sublists  Into  equivalent  project  sublists  required  several  steps.  First 
each  requirements  sublist  was  converted  Into  the  standard  form,  but  with 

Index  Integers  for  requirements. 

« 

Using! a  separate  model,  the  single  equivalency  statements  were 

Input,  searched,  and  compiled  In  groups  by  requirements.  Over  240 

equivalency  statements  were  used  In  the  translation  Index  for  this  study. 

« 

Table  18  contains  examples  of  the  Input  translation  equivalency  state¬ 
ments.  Table  19  contains  the  arranged  list  of  statements  stored  for 
access  In  structuring  the  translated  projects  sublist.  When  the  standard 
requirements  sublist  was  translated,  each  sublist  element  was  replaced 
by  all  of  the  projects  which  are  shown  to  be  equivalent  to  that  require¬ 
ment.  When  multiple  projects  replaced  one  requirement,  they  were 
considered  equal  and  Indifferent.  After  the  substitution  step,  the 
translated  projects  list  was  purged  of  all  duplicate  projects  after  a 
project  first  appeared  In  the  rank  order.  The  resultant  standard 
projects  sublist  was  aggregated  like  the  Input  projects  sublists. 

2)  Data  Sample  -  Table  18  In  the  previous  section, 
provides  a  sample  of  the  data  input  as  equivalency  statements.  The 
translation  index  has  approx imately  250  statements  for  this  study. 


TABLE  18.  TYPICAL  TRANSLATION  EQUIVALENCY  STATEMENTS 


L9 


C.  Computation  Outputs 


Aggregated  Rank  Orders 

The  R  &  D  prioritization  study  mas  calculated  as  cases.  In 
case  1,  the  laboratory  director  considered  each  technology  category 
project  as  unranked  and  the  sublists  are  not  synthetically  completed. 
Table  20  presents  the  final  R  &  D  projects  ranks  rank  order  for  Cases  1 
and  2.  In  Case  2,  the  laboratory  director  accepted  the  technology 
category  projects  prioritization  by  his  junior  managers.  Also,  In  Case 
2,  all  subllsts  were  synthetically  completed. 

2 .  Evaluations 

The  number  of  circular  trials  and  coefficients  of  consistency 
were  computed  for  Cases  1  and  2.  The  coefficient  of  concordance  was 
computed  for  the  Case  2  (completed)  Input  sublists.  Table  21  contains 
all  evaluation  data  for  this  study. 

3 .  Analysis 

The  outputs  In  Tables  20  and  21  show  that  the  conditions 
differing  between  these  cases,  namely  synthetic  completion,  made  only 
minor  adjustments  to  the  positions  of  the  alternatives  In  final  rank 
order.  The  sublist  rank  orders  were  all  tested  as  significantly  consis¬ 
tent,  but  the  output  PREF  Case  2  rank  order  was  0.05  significantly  in 
concordance. 

Except  for  Projects  55,  56,  and  57,  the  laboratory  director's 
preferences  fell  In  lexlgraphlc  order.  The  effect  of  the  aggregation 
In  Cases  1  and  2,  Table  20,  Is  Illustrated  by  the  relatively  high  ranking 
of  Projects  47,  27,  28,  83,  and  95  and  the  relatively  low  rankings  of 
Projects  3,  10,  23,  and  14  as  compared  to  the  lexlgraphlc  order. 


TABLE  20.  030NCLUDED) 


Rank 

Order 

Project  Index 

Rank 
.  Order 

Project  Index 

Case  1 

Case  2 

Case  1 

Case  2 

45 

•  8,53 

53 

68 

76 

76,94 

46 

62 

58 

69 

77 

77 

47 

58 

62 

70 

14 

72 

48 

60 

60 

71 

91 

14 

49 

54,61 

61 

72 

78 

78 

50 

63 

54 

73 

72 

91 

51 

64 

63 

74 

79 

73 

52 

65 

64 

75 

73 

79 

53 

66 

65 

76 

74 

74 

54 

67 

66 

77 

87 

87 

55 

23,68 

67 

78 

93 

90,92 

56 

69 

68 

79 

90 

93 

57 

70 

69 

80 

92 

55 

58 

71 

70 

81 

55 

56 

59 

81 

71 

82 

56,57 

57 

60 

89 

89 

83 

88 

88 

61 

82 

84 

84 

— 

— 

62 

80 

80 

85 

— 

— 

63 

84 

85 

64 

85 

86 

65 

86,94 

81 

66 

75 

75 

67 

50 

50,82 

TABLE  21.  EVALUATION  TESTS  BESDLTS 
RAD  PROJECTS  STUDY 
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D.  Study  Results 


t 


1.  Specific  Probless 

Several  problems  had  to  be  satisfied  for  this  study.  Firsts 

the  dimensions  of  the  model  arrays  had  to  be  resized  to  handle  the 

I 

unpurged  steps  within  the  requirements  translation  phase.  After  purging 
duplicate  projects,  the  100  by  100  dimensions  were  adequate.  Also,  the 
alternative  Index  dimensions  had  to  be  extended  because  the  requirements 
element  Indexes  were  different  and  In  addition  to  the  project  elements. 
All  dimensions  could  not  be  freely  enlarged  because  the  computer  model 
was  approaching  the  maximum  capacity  of  the  computer  facility. 

A  concern  of  this  study,  and  any  future  real  life  studies  of  this 
nature.  Is  for  the  decision-maker  to  establish  an  appreciation  and  feel 
for  the  sensitivity  of  the  aggregation  results  to  levels  of  weighting. 
For  example.  Is  a  weight  ratio  of  10  to  5  appropriate  between  the 
director's  weight  value  and  that  of  his  staff?  Possibly  10  to  7  or 
even  50  to  5  would  more  accurately  depict  the  decision-maker's  judgment. 
Table  22,  presenting  no  weight  versus  weighted  results,  was  used  to 
obtain  the  appreciation  for  the  weight's  sensitivity.  For  example,  the 
rank  positions  of  Projects  14,  23,  83,  and  89  differ  noticeably  between 
the  weighted  and  unweighted  versions  of  Case  1,  due  to  relative  effects 
of  sublist  ranks. 

2.  Methodology 

The  methodology  used  for  this  R  &  D  projects  prioritization 
study  was  quite  flexible  In  that  It  permitted  easy  changing  of  the  run 
option  controls.  The  model  was  effective  and  efficient  in  that  a  full 
run  for  this  study  required  less  than  one  minute's  computer  operating 
time  to  obtain  thorough  results.  As  Is  evident  in  this  95-project 
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study,  aggregation  of  several  long  rank  orders  would  be  isprsctical  and 
virtually  iaposslble  without  a  conyutor  nodel  and  a  large  computer. 
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TABLE  22.  EFFECTS  OF  WEIGHTS  R  &  D  PROJECTS 
STUDY  CASE  1 


Rank 

Order 

Project  Index 

Rank 

Order 

Project 

Index  1 

Case  1 

Case  3 

No  Hts. 

Case  1 

■9 

2 

1,2 

23 

20 

46 

1 

4 

24 

21 

44,51 

4 

7 

25 

25 

25 

^9 

7 

47 

26 

26 

26,59 

^9 

47 

45 

27 

29 

29 

^9 

5,12 

27 

28 

30 

30 

^9 

11,15,27 

12,28,49 

29 

31 

31 

8 

28 

5 

30 

10 

33,62 

9 

6 

3,6,15 

31 

33 

34 

10 

9 

11 

32 

34 

35 

11 

13 

17 

33 

35 

36 

12 

17 

83 

34 

36 

37 

13 

16 

95 

35 

37 

38 

14 

3 

9 

36 

38 

39 

15 

44 

13,22,24 

37 

39 

40 

16 

32 

16 

38 

40 

41 

17 

24 

32 

39 

41,83 

89 

18 

22 

18 

40 

42,51 

42 

19 

45 

19 

41 

43,46,95 

43 

20 

18 

10 

42 

48 

8 

21 

19 

20 

43 

59 

48 

22 

49 

21 

44 

52 

52 
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TABLE  22.  (CiXiCLUDED) 


Rank 

Order 

Project  Index  | 

Rank 

Order 

Project 

Index 

Case 

Case  3 

No  Wts. 

Case  1 

Case  3 

No  Wts. 

45 

8,53 

53 

68 

76 

86 

46 

62 

23 

69 

77 

77 

47 

58 

58 

70 

14 

78 

48 

60 

54 

71 

91 

79 

49 

54,61 

14,60 

72 

78 

91 

50 

63 

61 

73 

72 

73,87 

51 

64 

63 

74 

79 

74 

52 

65 

64 

75 

73 

92,93 

53 

66 

65 

76 

74 

90 

54 

67 

66 

77 

87 

55 

55 

23,68 

67 

78 

93 

56 

56 

69 

68 

79 

90 

57 

57 

70 

69 

80 

92 

88 

58 

71 

70 

81 

55 

59 

81 

50 

82 

56,57 

— 

60 

89 

71 

83 

88 

— 

61 

82 

60 

— 

— 

62 

80 

81 

— 

— 

63 

84 

82 

64 

85 

84,94 

65 

86,94 

75 

66 

75 

85 

67 

SO 

72,76 

3 .  Study  Conclusions 

This  prlorltlzstlon  study  of  R  &  d  projects  hss  deaonstrated 
that  the  model  can  be  used  effectively  to  aggregate  long  rank-ordered 
sublists  of  R  &  D  projects.  It  further  demonstrated  Chat  the  model 
can  convert  and  translate  rank-ordered  sublists  of  product  requirements 
Into  equivalent  sublists  of  R  &  D  projects  which  are  then  aggregated 
with  the  other  sublists  of  R  &  D  projects. 

The  study  did  show  that  deliberate,  subjective,  and  problem  peculiar 
decisions  must  be  made  In  preparing  the  Inputs  and  especially  In  selecting 
the  options.  As  previously  mentioned,  the  desired  level  of  weight 
sensitivity  must  be  established.  When  categorized  data  exist,  the  Judge 
who  ranks  the  categories  must  determine  whether  he  Is  Indifferent 
between  projects  within  a  category  or  whether  he  will  accept  the  wlthln- 
category  project  ranking  of  someone  else,  such  as  a  Junior  manager 
specializing  In  that  category.  If  Judge  self-evaluation  Is  not  used, 
the  Judge  who  develops  a  partial  sublist  must  decide  whether  be  Is 
Indifferent  between  the  remaining  projects.  If  he  Is  Indifferent,  he 
can  choose  for  the  model  to  complete  the  sublist  synthetically.  If  he 
Is  not  Indifferent  between  the  remaining  unranked  projects,  then  he 
probably  should  not  have  the  sublist  synthetically  completed. 
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CHAPTER  VIII.  SU>&{ARY  AMD  REC0MMEMDATI(»1S 
A.  Concluding  Sunnary 

The  literature  search  for  this  research  determined  that  few  articles 
have  been  written  related  to  R  &  D  resources  Banagement  prioritization 
through  an  ordinal  aggregation  process.  But  there  Is  an  extensive  body 
of  literature  on  the  more  general  field  of  social  choice  for  political 
science,  social  science,  and  economics.  This  research  analyzed  this 
large  body  of  material  and  presented  the  literature  sumnary  by  structuring 
It  into  time-phased  thrust  areas. 

Numerous  methods  have  been  documented  to  aggregate  rank-ordered 
subllsts.  Many  of  the  methods  have  characteristics  that  prohibit  their 
use  for  the  objectives  of  this  research.  Many  of  the  potentially  usable 
methods  employed  a  majority  rule.  Only  a  few  pieces  of  literature 
compared  aggregation  methods,  and  even  fewer  articles  evaluated  and 
selected  between  methods  for  specific  applications.  This  research 
Identified  seven  promising  majority-rule  methods  and  classified  them 
for  their  usefulness  in  the  specific  application  of  multiple  judge,  no 
feedback,  ordinal  prioritization  of  R  &  D  projects. 

A  model  was  developed  and  coded  on  a  large  computer  to  accomplish 
the  sublist  aggregation,  weighting,  hierarchical  conversions,  require¬ 
ments  translation,  and  results  evaluations.  The  coded  model  has  been 
verified.  Validation  has  been  successfully  performed  against  46 
examples  from  the  literature  of  which  13  are  Included  in  this  report. 

The  model  was  then  demonstrated  for  an  extensive  R  &  D  projects 
prioritization  study. 
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Fuzzy  set  rank-order  nethodology  was  briefly  explored  and  added 
to  the  model  for  an  alternative  final  aggregation  rank  ordering.  The 
methodology  csmployed  was  too  insensitive  for  many  of -the  cases  computed. 
The  fuzzy  set  method  would  rank  many  alternatives  as  indifferent  when 
the  other  three  methods  developed  preference  orders  between  the  same 
alternatives . 

0  B.  Research  Accomplishments 

In  reflection  on  the  research  reported  in  this  report  and 
Dobbins  [13,  14],  several  findings  and  accomplishments  are  apparent. 
First,  this  research  demonstrated  how  a  field  of  knowledge,  namely, 
social  choice,  possesses  applicability  to  engineering  management  situa¬ 
tions.  This  phenomenon  stimulates  the  question  of  which  other  fields 
of  knowledge  are  being  actively  developed  in  another  area  that  could  be 
of  direct  benefit  to  engineering  management. 

This  research  demonstrated  the  practicality  and  limitations  of 
several  majority- rule  methods  that  can  be  used  to  aggregate  ordinal 
rank  orders.  Although  extensive  theoretical  research  has  strived  and 
generally  failed  to  find  aggregation  methods  that  always  give  intransi¬ 
tive  results,  for  the  realistic  rank  order  problems  examined,  intransi¬ 
tivity  was  not  an  impediment. 

Specifically,  this  work  has  shown  that  diverse  and  complex  R  &  D 
management  priority  lists  can  be  aggregated  into  a  useful  single  rank- 
ordered  list.  A  real  life  example  was  studied  where  13  sublists  with 
95  projects  and  44  requirements  were  successfully  aggregated. 

Finally,  this  researcher  has  been  rewarded  by  the  sense  of  accom¬ 
plishment  of  progressing  through  all  of  the  steps  of  a  full  research 
project. 
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C.  Reconmendatlons  for  Future  Investigations 

1 .  Methodology 

Through  the  conduct  of  this  research,  certain  methodology 
questions  recurred  which  were  interesting,  but  outside  of  the  specific 
scope  of  the  research. 

Black  [4]  developed  a  concept  he  named  the  slngle-peakedness 
criteria  for  rank-orders.  Theoretical  studies  have  shown  slngle- 
peakedness  to  be  a  condition  that  can  be  directly  associated  with  transi¬ 
tive  rank  orders.  But  none  of  the  literature  located  gave  specifics  on 
the  application  of  the  slngle-peakedness  criteria  to  rank  order  aggre¬ 
gation  problems.  Logic  can  be  developed  to  evaluate  the  monotonic 
characteristics  along  an  order  as  compared  to  a  reference  sequence  of 
the  alternatives.  What  is  not  obvious  is  how  one  can  efficiently 
determine  the  reference  sequence  that  will  allow  all  sublists  to  be 
single-peaked.  If  a  method  is  not  developed,  then  all  sequence  coinbl- 
natlons  of  the  alternatives  must  be  evaluated  before  an  answer  can  be 
given.  Slngle-peakedness  methodology  is  further  aggravated  by  indiffer¬ 
ences  (ties)  or  partial  sublists. 

A  promising  majority-rule  method  was  attributed  to  Copeland  by 
Goodwin  in  Thrall  [41] .  After  Intensive  library  and  personal  inquiry 
to  Copeland's  family,  a  copy  of  Copeland's  memo  has  not  been  obtained. 
Another  recommended  topic  for  further  study  would  be  to  search  further 
to  obtain  a  copy  of  Copeland's  memo  and  determine  why  several  papers  in 
the  literature  stated  that  it  was  limited  to  sublist  rank  orders  with 
no  indifference  (ties) . 

The  fuzzy  set  rank-order  methodology  which  was  Included  in  the  model 
frequently  gave  rankings  that  were  too  insensitive  to  differences  in 


132 


rank  between  two  or  more  alternatives.  Further  study  could  determine 
what  number  of  alternatives  and  nuiid>er  of  judges  would  make  fuzzy  set 

rank  orders  adequately  sensitive. 

i  '! 

2.  Model  and  Computer  Code 

Several  areas  also  exist  for  further  research  work  to  Improve 
the  modeling  and  computer  coding  for  the  aggregation  of  rank  orders. 

The  dimensions  of  the  computer  code  arrays  are  limited  by  the 

computer  capacity.  With  no  significant  changes  In  the  model,  the  present 

% 

100  X  100  dimension  limit  could  be  enlarged  to  125  x  125  or  perhaps 
140  X  140,  but  little  further.  The  beneficial  solution  would  be  a 
computer  code  that  was  not  dimension  limited.  The  approach  might  be  to 
develop  a  computer  code  that  will  progress  through  very  large  matrices 
one  section  at  a  time  until  all  sections  are  computed.  This  type  of 
modification  might  permit  the  model  from  this  research  to  be  used  for 
aggregation  of  rank-ordered  preferences  of  segments  of  the  population. 

The  present  model,  to  minimize  data  storage  requirements,  does  not 
hold  Input  sublist  data  as  the  computations  progress  through  the  arrays. 
This  space  saving  requires  that  all  data  be  re-input  for  each  problem 
even  If  only  a  single  control  value  changed.  Again,  extended  space 
capacity  could  remedy  this  Input  data  repetition  requirement.  Further 
research  might  find  other  remedies. 

The  COMPARE  subroutine  used  Kendall's  concordance  tests  to  evaluate 
pairs  of  final  aggregated  rank  orders.  Kendall's  concordance  method  was 
necessary  where  more  than  two  rank  orders  were  evaluated.  But  there  were 
other  methods,  such  as  Kendall's  Tau  method,  that  could  be  considered 
where  there  are  only  two  rank  orders.  An  Investigation  could  determine 
If  Kendall's  concordance  test  should  be  replaced  for  these  final 
comparative  tests. 
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In  conclusion,  the  research  for  this  report  Is  believed  to  be 
a  contribution  to  the  field  of  engineering  management  knowledge.  But 
In  most  research,  the  development  of  knowledge  has  revealed  additional 


questions  to  be  answered 


APPENDICES 
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Appendix  A.  FUNCTIONAL  FLOW  DIAGRAM 

The  functional  flow  diagram  for  the  aggregation  computer  model  la 
presented  in  Figure  A-1.  The  P  term  repeated  In  the  flow  means  PRINT. 
It  concerns  the  Information  about  the  steps  In  the  process. 


functional  flow  diagram. 


Figure  A-1.  CContinued) . 


Figure  A-1.  (Continued). 


(Concluded) 
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Appendix  B.  INPUT  INSTBUCTIONS 

Input  Instructions  fox  lending  the  controls  end  data  Into  the  aodel 
are  presented  In  Table  B-1. 
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TABLE  B~l.  TECHNOLOCT  PLANNING  PRIORITIES 
^nput  Requirements 

Card  Type  1:  Header  -  Name  of  priority  group 
Col  01-80 

Card  Type  2:  Control  card 

Col  05-05  -  NWT  -  Wei^t  type  (1-8) 

(see  Dobbins  [14]  for  descriptions) 

Col  10-10  >  NC(BIP  »  Complete  all  matrices  if  nonzero 
Col  15-15  ■  NPTYPl  ■  Type  of  calculation  for  frequency  matrix 
0-0,  .5,  1 
1  -  -1.  0.  1 

Col  20-20  -  NPTYP2  -  Type  of  calculation  for  preference  matrix 
0-0,  .5,  1 
1  -  -1,  0,  1 

Col  25-25  -  MATR  -  Self  evaluation  key 
0  -  No  self  evaluation 

1  -  Self  evaluation,  complete  matrix 

2  -  Self  evaluation,  threshhold,  reduced  matrix 

Col  26-30  -  THLD  -  Percentage  level  under  which  elements 
are  discarded 

Col  35-35  -  NPRINT  -  PRINT  control 
0  -  Print  all 

1  -  No  print  of  sublist  frequency  matrices 

2  -  No  print  of  sublist  frequency  matrices  or 

weighted  sublist  frequency  matrices 

3  -  Same  as  NPRINT  -  1  plus  no  print  FUZZY 


4  *  Same  as  NPRINT  -  2  plus  no  print  FUZZY 

5  *  Print  only  Input  and  output 

6  -  No  print  FUZZY 

Card  Type  3:  Input  Type 

Col  S  ■  JELE  >  Element  code 
0  -  End 

1  >  Requirements 

2  ■  Projects 

Col  10-20  *  NELE  *  Element  type  name 
Card  Type  4:  1  -  NBR  NBR  *  Number  of  requirements 

Col  03-05  >  K  ••  Element  number  -  Number  between  1  -  NBR 
Col  11-30  -  NAM  *  Element  name 
Col  31-40  ■  WHI  ■>  Row  weight 
Col  41-50  ■  KAT  »  Category 

Terminate  Element  cards  with  "END"  In  Col  11-13 

A.  Element  Number  and  name  are  required.  If  the  weight  or  category 
factors  are  blank,  they  are  assumed  to  be  0, 

B.  If  a  weight  type  is  assigned  In  Card  Type  2,  a  weight  factor  must 
appear  on  the  project  card.  If  the  projects  are  not  weighted, 
but  the  judges  are,  then  use  a  one  (1)  on  each  card. 

Categories  are  used  only  in  the  cases  where  one  or  more  of  the 
evaluators  uses  a  judge  conversion  factor  of  9,  10,  11,  or  12. 

In  which  case  the  CATEGORY  (KAT)  groups  certain  projects  or 
requirements  together.  If  the  projects  within  a  category  are 
ranked,  they  must  appear  in  their  ranked  order. 

Element  Nund>er  1  2  3  4  5  6  7 

Element  Name  A  B  C  D  E  F  G 

Element  Weight 

Element  Category  1  122233 

The  foregoing  example  implies  that  1>2,  3>4>5,  6>7 
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The  final  order  of  the  requirements  %fould  depend  upon  the  ranked 
or  unranked  state  of  the  categories.  If,  however,  the  requirements  are 
specified  unranked,  then  the  foregoing  example  would  laq>ly 

1-2,  >4-5,  6-7 

and  again  the  final*  order  of  the  requirements  would  depend  upon  the  ranked 
or  unranked  condition  of  the  categories. 

Sublist  Data  Card  Sets 

Card  Type  5:  Card  1 

Col  01-10  -  Judge  -  Name  of  Judges  or  office  making  rank 

Col  14-15  -  JCONV  -  15  Type  of  project  conversion 

(see  Appendix  B  for  descriptions) 

Col  16-20  -  WTJ  -  Weight  factor  of  Judge. 

Col  21-25  -  ISEM  -  lOOZ  weight  factor  for  self  evaluation 

A.  Judge  Name  -  Name  of  evaluator  must  be  present.  If  the  JCONV 
or  WJT  left  blank,  they  are  assumed  to  be  0. 

B,  If  the  JCONV  is  specified,  the  program  looks  for  specific  data 
in  Card  2  -  Free  format  sublists. 

JCONV  Input  Requirement 

1  Normal  input 

2  Reduced  stibllst 

3  Input  reduced  sublist.  Program  will  complete 

it  at  end  with  equal  elements  all  less  than 
the  last  given  element. 

4  Input  reduced  sublist.  Program  will  complete 

SL  at  the  beginning  with  equal  elements  all 
greater  than  the  first  given  element. 

5  Input  rating  values  in  real  numbers  given  in 
the  order  of  project,  e.g., 

ABC 

1,  2,  3,  etc.  Program  will  arrange  projects 
in  order  of  highest  to  lowest,  setting  equivalent 
elements  equal. 
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Input  Julian  date  of  projects  In  order  of 
projects.  Program  will  arrange  projects  In 
order  of  soonest  to  latest,  setting  equivalent 
elements  equal. 

7  Input-3  Freeform  subllsts 

Card  A  Key  element 

Card  B  Secondary  array  to  be  Inserted  Into  primary 
array  after  key  element. 

Card  C  Primary  array  -  Program  Inserts  secondary 

array  In  primary  array  checking  for  duplication 
of  each  element. 

8  Input-3  Freeform  subllsts 

Card  A  Key  element 

Card  B  Secondary  array  to  be  Inserted  Into  primary 
array  before  key  element. 

Card  C  Primary  array  -  Program  Inserts  secondary  array 
In  primary  array  checking  for  duplication  of 
each  element. 

-12  Categories  must  be  specified  In  project  cards. 

9  Input  ranked  categories  -  Categories  must  not 
be  equal.  Program  checks  for  ranking,  then 
groups  ranked  requirements  by  category. 

10  Input  unranded  categories  -  Categories  must  be 
equal.  Program  checks  for  ranking,  tiien  groups 
ranked  requirements  by  category. 

11  Input  Ranked  Categories  -  Categories  must  not 
be  equal.  Program  checks  for  ranking,  then 
groups  unranked  requirements  by  category. 

12  Input  unranked  categories  -  Categories  must  be 
equal.  Program  checks  for  ranking,  then  groups 
unranked  requirements  by  category. 

(If  categories  are  Improperly  Input,  an  error 
message  Is  written  and  the  subllst  Is  dropped 
from  calculations.) 
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C.  If  weight  type  factor  appears  on  Card  Type  2,  a  weight  factor 
must  appear  on  the  evaluator  card.  If  projects  are  weighted, 
but  not  the  judges,  then  use  a  one  (1)  on  each  card. 

Card  Type  6:  Card  2  -  Free  format  subllst  ranks  by  Project  Number. 

Sequence  Indicates  preference,  prefix  with  minus  to 
Indicate  equal.  Terminate  list  with  a  *.  Follow  special 
rules  for  specific  JCONV  outlined  above. 

Card  Type  7:  Self  evaluation  of  expertise  In  the  technical  field  of 

each  Input  element.  These  ratings  must  be  between  0  and 
ISEH  In  Che  element  index  order.  Use  Card  Type  7  only 
when  MATR  -  1  or  2  (Col  25  Card  Type  2) .  This  Is  a  free 
format  list  of  Integers  terminated  with  an  *. 
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Appendix  C.  CODE  LISIIDG 


The  FORTRAN  IV  code  listing  for  the  model  computer  code  Is  presented 
in  Table  C-1.  A  more  comprehensive  description  of  the  computer  code. 
Including  exanqples  can  be  found  In  Dobbins  [14],  a  report  of  the  US  Anqr 
Missile  Laboratory. 
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TABLE  C-1.  FORTRAN  IV  CODE  LISTING 


nnnatMS  T«/T«  !tPTal 


PT*« 


I 


n 


1*; 


an 


?a 


1« 


an 


a« 


SO 


ss 


aODGaOM  OnMSI>lS(lMPUT*nUTPltT.raPCSaTNPUTtT*PESBOUTPJT«T«PF«l 

r 

r  TFCHNOLOSV  PLOMMINA  PPIPaTTTFS 

C 

c  jop  uos  FD  nnpRivs  /  w.  jnyrs 

c 

c 

r  rnvaoN  /cnara/  npo  Raaf ip.snni •A(i»n«iooi »4mnnt f wjnoil 

a  «  JN*MEIIOIt«MStFF(l(in.JSIIHL<10A»l»l» 

C0PanN/HFLF/'<W.INr)l300>*tCaTnBat  •ROMIEi  •JCMECKOnoi  •TTTtlMtX.Jsax 
COManM/aaNK/LlSTC(]OOtSI*I.IST(l»OI*L*R(3> 
rOaMOM/IOr)/-4rAOFP(n|,«|OTCOM«>|PTVP|.NPTVPP«i«F)l7«NPRINT«JTte 
COMaOiil/  /aaTa<THlO<SFM(lon),r«Q(ino»ien 
c 

niMESSIOH  Sja(200l*  aOJPdOO) 
niMEVStOM  tPRFF(2,iA0l«JPREF(in0)«JP*MKnnAI 
oar*  NOT/"  >"/t  NFO/»  a"/ 
oar*  NOasM/"— — «/ 
m  PFaD(S««n?i  "EanFR 
40?  FOPaart  saio  I 

IFI  FOF(  S  »  .NF.  n.o  »  STOP  7TT 
/OITFCO.QOOt  HFaOFP 

ooo  rooaaT<«lTCC-»NOLOOV  PLANNING  P4tORITlES"»10**44IO//» 
r 

r  NEAO  INPUT 

call  input 

c. 

r.  COMPUTF  FPEGUFNCV  NATPI*  FD4  EACH  SUP-LIST 

call  fpfo 

r 

c  PRINT  sjNNen  freguency  natpix 

IF  (NPRINT.3E.3.ANn.NPPINT»Lr.TINFU7B| 

IF  INPRINT. EG. 5»  00  TO  SOS 
IF  INNT.flC.l. FNO.NNT.lt. SI  GO  TO  #00 
IF  UTIF.FO.I)  GO  TO  aSO 

r.  PRINT  SlINPFO  FREO  MATRIX  -  JTIEbO*  NO  NEIGHTS 
MPITE (6.4401  "EAOES*  (  J  .  Jal.NPR  I 
440  format  ("ISJNNEO  FREQUFNCV  MAT4IXN.P0X.S61 A.//"  POPOA  ADJN. 

•(T2I.10ISII 

NPITE(0.94T)  (  NOASN.  Jal.NSP  ) 

9a7  FORMAT  (IX.TPO.ISAOI 
00  TO  SOS 

500  IF  (JTIE.FO.II  60  TO  SOI 

C  "RINT  SUMMFO  FOEO  MATRIX  -  JTIEbO.  M/WEieMTS 

"RITE  (6.9491  HEADER. (J.JbI.nRRI 

449  FORMAT  ("IS'JMMEO  FREQUENCY  MATRI X«.POX.SAI 0//."  EOllTV  EOUIVa. 
a/."  RORDA  AOJ  ROROA  AOJ  ".(TS4.ISIMI 

WRITE  (O.RSO)  (NOASH. JsI.NPRI 
4S9  POPMAT  (IX.TSa.lSAPI 

GO  TO  505 

r  PRINT  SUMPF.J  FREO  MATRIX  -  JIIE*!.  W/WFIGMTS 

501  WRITE  (6.990)  HEADER.  (J..IB1.N44) 

990  FORMAT  ("IS  IMMEO  FREQUENCY  MATRIX". POX. RA1  (*.//"  JUDGE  INDIFFERENCE 
a  EXISTS".//"  EOUtW"./"  AOJ  40R0A  AOJ  GOROA". (TP4. IRTSI I 

WRITE  (6.491)  (NOASH.  JBI.NRR) 

991  FORMAT  (1X.TP4.1GA6) 

00  TO  SOS 
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TABLE  C-1.  (CONTINUED) 


ea0««4«*  OOMBTNS  74/74  OPT«| 


FTIil  *.6*4)« 


40 


•■4 


70 


74 


40 


•  4 


40 


44 


100 


104 


no 


C  bhiMT  SUMMPO  FREO  PATAU  •  JTT'BI*  MO  HEIGHTS 
440  HAITE  (4t44?)  HEAAFA* | J. Ja1 *H44I 

QAA  roAHAT  («1SU4ME0  PAEOUEHCV  HAT4T AM,a0X*AA10»//h  JUDGE  IHOIFPFAPHCE 
*  E»ISTSH»//“  AOJ  AOAGAN,(Tl4«lRtOn 
HAITE  (4i444t  tH0ASH«Js|*N4A) 

944  P0P4AT  (UtT  14*14401 
404  CONTINUF 

DO  0?a  I  ■  1*VGA 

C  ADADA  COUHT  AND  ADJ  OOADA 

AIIMA  a  0.0 
no  010  J  a  l.HGA 
010  41IHA  a  SUMO  *  A|I«J1 
4UMC  a  0,0 
DO  015  J  •  t.HOR 
014  4(IhC  a  SIIHC  ♦  AIJ.II 
AOJ  ■  SUMA  •  SUMC 
SUHin  ■  SIJ4R 
A0J4(I)  a  ADJ 
IF  <HARtNT.ea.4l  GO  TO  420 
IF  IHhT.GE.I.AHQ.HHT.LT.OI  go  TO  OOO 
80  TO  0?4 

000  IF  (JTlE.EO.lt  GO  TO  001 

HAITE  (4.441  >  SUHA,A0J.4UMA/4,Ar>J/«.I»  I A  1 1 ,  J)  .  Jaf  .HSA) 

941  FORMAT  (lX*4rT.l.lX«n.lT44*MT  ».14F4.1)) 

SO  TO  020 

401  hAITE  (0.944)  A0J*A0J/4..I. |A(I.J)*Jal«NBA) 

994  format  (U.?(FT.I.iX).1.T»(T2?."T  ».14P0.1)) 

GO  TO  020 

024  COHTIMUE 

IF  (JTIE.E0.1)  60  TO  002 

4AITE (0.944)  SUMA.  AOJ*  I*  (  A(I*J)  *  JBl.MGR  t 
944  FORMAT  ( I X* 2FT.1 * U* U ( T>2*»I  »*1AFG,1))  I 
SO  TO  020 

002  HRITE  (0*945)  AOJ.I * ( A( 1* J) * Jal .HGA) 

995  aORmAT  (lX,rT.l,Ia*2X*(n3*HI  m*1GP0.1)) 

620  CONTINUE 

IF  (JTIE.EO.l)  GO  TO  430 
C 

CALL  order  (  ••40ROA  »  *  N4R  *  SUM  ) 

430  CONTINUE 

call  OAOEA(  "AOJ  "  *  mgr  *  AOJR  ) 

C 

C  COMPUTE  PREFEAFNCE  MATRIX 

CALL  AREF 
C 
C 
C 

C  4TPASS  IF  HETSMTEO  ANO  INCOMPLETE 
IE  (MOTCOM.ME.O)  go  to  14 
IF  <MFU7.NE.0I  GO  TO  9 
CALL  FU7Z7 
9  CONTINUE 
C 

IF  (NPAINT.Ea.S)  GO  TO  I* 
call  CONCOA 
14  call  compare 
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TABLE  C-1.  (CONTINUED) 


BI»nR44w  T4/T*  3BTbI  fT*!  4.4*A^« 


11^  C 

C  B»IMT  BBRFEBCNCC  Sn**M4*Y 

14  CONTI HUE 

1)0  9A0  Jal*N4B 
KaLl9TCU.Fi 

IFO  L«M"9|K» 

.IBANK  Ul  at 
JPBCr UiaNAT 

IF  IK.LT.BI  JBBFFUiamCq 
anO  CONTINKF 

1F9  JPBFFa»«a  • 

00  PIP  lal.NPB 
on  410  Jal.NPP 

IF  (IAP4aI9TC(I.2)l.eQ.INIIUII  IPRFFII.IlaNANCIl.JI 
IF  (lABS(LtSTCfl.FI).FQ.INn(JII  IPRCF 12. 1 I bNANC (?. Jl 
IIP  «1P  CONTINUC 

KBITC  (P.B44t  HFARFR 

44K  F0R4AT  (1N1.T40.  PAl 4. ///TApNPBFFCRFNCC  RANK  OROCR  SUNNART"///! 

NPITF  (P.«4>l  UPPFFlIi.JBANKin.Ial.NRR) 

OAF  FOPNATI*/  CLCNCNT  INDCK  rank  ARDFR*./  2P(A2.IS)i 
144  KRITE  U.BAPt 

BPP  FORNAX  I//TO.  NPANKM.TX.NPRtJECTPi 
Jal 

00  PFO  lal.NOB 

IF  M.NC.I.ANO.LISTCII.Ft .RT.PlJaJ.I 
140  KBITE  lA.BPIi  J.lPREFll.n.tPBCFIF.n 

4FP  CONTINUE 

BPl  FORNAX  (//ri0»I3«4X.?A10) 

00  TO  IP 
FNO 


CABO  NB,  SEWFBITV  OCTAltN  0140M041S  OF  PBOPlCN 

P9  I  29  CO  49  SEPARATOR  NISSTNO.  SEPARATOR  ASSURED  MFRF. 
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TABLE  C-1.  CCCaiTINUED) 


Tk/T4  nPTml 


INPUT  SJ^POUTToir 


PTN  4.P*«99 


KCV 


V*«T«NUE 

ANR  RISCRIPTIRN 

SUBROUTINE 

SUM 

•  HR090AH  count  tariablc 

DOBBINS 

tnjR 

■  "ARJ  AOB9AH  COUNT  VARIABLE 

OOBBIMS 

nnh 

•  NUMBER  RE  ClCNENTB 

INPUT 

< 

•  INDEX  RE  ELEMENT 

INPUT 

PHI«NII 

•  ELEMENT  MEISMT  FACTOR  (ROW) 

INPUT 

•CAT 

•  CATESORV  'FlENENT  PRRUP 

INPUT 

Nwr 

■  WEI8NT  TTRE-OVERALL  WEIBMTIN6 

INPUT 

factor 

INPUT 

NcnMP 

•  Flag  fob  completion 

INPUT 

NPTTPl 

«  erfrufnct  matrix  type  conversion 

INPUT 

■KPTVPP 

•  PREFERENCE  MATRIX  TYPE  CONVERSION 

INPUT 

MAT9 

•  UU06E  SELF  evaluation  FLAG 

INPUT 

THLO 

■  percent  lfvel  under  which  the 

INPUT 

ELF.MENTS  ape  discarded 

INPUT 

nprint 

•  print  control  'LAO 

INPUT 

jiiosr 

»  NAMf  OF  evaluator 

INPUT 

JCANV 

•  JUOOC  CONVERSION  KEY  TO  ARRANRE 

INPUT 

DATA  into  STANOARO  FORMAT 

INPUT 

NTJsNJJ 

■  UUORE  WEIGHT  »ACTOR  (COLLUM) 

INPUT 

rsfN 

■  100  PERCENT  weight  »ACT0R  *-0  SEL* 

INPUT 

evaluation 

INPUT 

JFLE 

■  CORE  FOR  input  TVPEtE.fi. tPROJECTS 

INPUT 

OR  REOUIREMENTS 

INPUT 

ESP 

«  JUDGE  SELF  EVALUATION  PERCENTAAE 

INPUT 

EWdLOE 

■  FORMATED  OUTPUT  FROM  PRAM 

INPUT 

C 

■  KET  ELEMENT  FOR  JCONV  p7  DR  R 

INPUT 

IM«X 

•  maximum  number  of  elements 

INPUT 

two 

•  ARRAT  OF  INDEXES  OF  ELEMENTS 

INPUT 

ITT 

■  total  number  of  projects 

INPUT 

JN*X 

■  maximum  number  of  judges 

INPUT 

JNXME 

P  ARRAY  OF  JUDGES*  NAMES 

INPUT 

jpref 

m  ARRAY  OF  p*S  AND  >*S  FOR  EACH  RANK 

INPUT 

ORDER 

INPUT 

JRANK 

p  ARRAY  OF  rank  ORDER 

INPUT 

JSUSU 

p  ARRAY  OF  ALL  JUDGES'  RANK  ORDER 

INPUT 

NAME 

p  ARRAY  OF  ELEMENT  NAMES 

INPUT 

NEUE 

P  flag  TO  omit  FUEZY 

INPUT 

NJ 

p  NUMBER  OF  JUDGES 

INPUT 

NSIZE 

P  ARRAY  OF  THE  NUMBER  OF  ELEMENTS 

INPUT 

EACH  JUOGF  ranked 

INPUT 

NV 

•  number  of  fle rents  found  by  pram 

INPUT 

IM  EACH  RANK  ORDER 

INPUT 

SEM 

p  ARRAY  OF  SFLFEV  EVALUATION  VALUES 

INPUT 

READ  BY  pram 

INPUT 

NI 

p  array  of  element  weights 

INPUT 

p  array  of  judge  weights 

INPUT 

LIST 

p  array  of  elements  for  composite  rank 

ORDER 

ORDER 

ORDER 

LISTC 

P  ARRAY  OF  ELEMENTS  FOR  THE  THREE 

ORDER 

COMPOSITE  rank  orders 

ORDER 

LAB 

p  NAME  OF  THE  COMPOSITE  RANK  ORDERS 

ORDER 

TABLE  C-1.  (CONTINUED) 


T«/T«  OPTal 


rTN 


c 

cs 

• 

CHI-SQUARE  STATISTIC 

RREF 

c 

0 

• 

kpnoall  0  bnr  of  circular  triaos  in 

RREF 

an 

c 

RRFF 

RREF 

c 

OS 

• 

VALUES  OF  KENOALL  0  AT  RAM6E  LEVELS 

RREF 

r 

LAR 

m 

LAREL  FOR  RANRE  OF  NFNRALL  0 

RREF 

c 

ONU 

• 

OEOREES  OF  FREEOON 

RREF 

c 

NF 

• 

NUMRER  OP  FRACTIONAL  SUMS 

RREF 

AR 

c 

P 

• 

RRARARILITV  THAT  RANK  IS  NOT  CONSISTANT 

RREF 

c 

"TEST 

■ 

FIXED  CRITICAL  VALUE  OF  R 

RREF 

c 

FETA 

« 

COEFFICIENT  OF  CONSISTANCV 

RREF 

c 

AII.JI 

• 

NORMALIFEP  FREOUENCV  MATRIXt  R 

FUZEY 

c 

TRACEC 

« 

SUM  OF  THE  major  01 agonal  OF  AlltJI 

FUZZY 

TO 

c 

MATRIX  SOIIAREO 

FUZZY 

c 

TRACT  E 

« 

SUM  OF  THE  MAJOR  OIARCNAL  OF  A(I*J1 

FUZZY 

c 

MATRIX  •  A(I*JI  TRANSROSFO 

FUZZY 

c 

ER 

• 

average  futtimess  in  r 

FUZZY 

c 

CR 

• 

AVERAOE  CERTAINTV  IN  R 

FUZZY 

TR 

c 

S 

■ 

SUM  OF  DEVIATIONS  SQUARED 

CONCOR 

c 

0 

• 

KENOALLS  coefficient  of  CONCORDANCE 

CONCOR 

c 

RRAR 

a 

mean 

CONCOR 

c 

R 

a 

RRORARILITV  OF  RANK  ORDER  CDNCDRDANCE 

CONCOR 

c 

NELE 

a 

NUMRER  OF  ENTRIES  ON  INDEX 

REQUIRE 

an 

c 

f**a 

Rk 

TN»UT  DATA 

C 

c»# 

CAROS  1  THRU 

NSR  a  NUMRER  OP  RROJECTS 

c 

COL  43-OS  1 

■  K 

a  ELEMENT  NUMRER.  NUMRER  RETMEEN  1  ANO 

NOR. 

AS 

c 

11-30  < 

m  NAM.  element  name. 

c 

30-40  a 

VHI 

a  ROM  MEISHT  FS.O 

c 

40-S0  a 

SAT 

a  CATEGORY  IS 

C  TfRMlNATC  PROJECT  CMOS  WITH  »ENO“  IN  COLS  >I-|S. 
C 


«S 


mo 


ms 


im 


c** 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c 

c»* 

c 


CONTROL  CORO 

COL  RS-OS  •  MoT  •  WCI9MT  TVRS  I  THRO  S 
0  •  HO  MEtOHTS 

COL  iO-lO  ■  HCOHRa  COHRLCTE  *LL  HOTRICES  If  HOH  EERO 
COL  tS«lS  ■  HRTVRl  .  PREQUEHCV  HRTRtX  TVRF  CONVERSIOH 
COL  2R<-20  >  HOTVR2  •  RREFFREHCE  NOTRIX  TYRE  CONVERSION 
•  >  Ot.Stl 
1  >  •1»0*I 

COL  ?S-?S  ■  HOTR  -  SELF  E¥*LU»T10N  RET 

0  •  NO  SELF  EVALUATION 
I  ■  SELF  evaluation  COhRLCTE  MATRIX 
Z  •  SELEEVALUATION  thresh  hold  reouced  hatrtx 
?6-30  ■  THLO  >  level  UNOER  MHICH  THE  ELEMENTS  ARE  OISCARDCO 
COL  3I-3S  ■  NRRINT  -  NO  RRINT  RET 
f  •  RRINT  ALL 

1  •  NO  RRINT  OF  SUR'LIST  FREOUENCV  MATRICES 
F  •  NO  RRINT  OF  SUR-LIST  FREOUENCV  MATRICES 
OR  VETOHTFD  SUR-LIST  FREOUENCV  MATRICES 
3  >  SANE  AS  NRRINT  ■  1  RLUS  NO  RRINT  FUEZV 
A  •  SAME  AS  NRRINT  aZ  RLUS  NO  RRINT  FUZZV 
R  a  RRINT  ONLY  INRUT  AND  OUTRUT 
a  a  NO  RRINT  FUTEV 

SUR-LIST  DATA  CARO  SETS  •• 

CARO  I 
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TABLE  C-1.  (CONTINUED) 


T*/7*  0PT*1 


rTl 


COL  01-10  ■  JUOOe  ■  NAXe  of  JUOSE  OO  OFFICF  MAKlNff  OO^K  *10 

COL  lA-19  ■  JCONV  ■  TYPE  OF  .bpOJECT  CONVEPSTOM  IS 

COL  10-20  ■  OTJ  •  MCISHT  FACTOP  OF  JUDSC  FS.O 

COL  21-2S  a  ISEM  -  100  PEPCENT  WEIOHT  FACTOP  FOP  SELF  EVALUATIOY 

CAPO  2  a  FPEE  FORMAT  SUP-LIST  PAMKS  RV  PROJECT  NUMBER. 

SE9UENCE  INOICATFS  PREFCPENCEt  PREFER  alTH  MINUS  TO 
INOICATE  EQUAL.  TERMINATE  LIST  PITH  AN  •. 

CARO  T  a  free  format  SELF  EVALUATION  LEVELS.  ONE  FOR  EACM  RANKED 
ITEM  IN  ORDER.  MUST  BE  LESS  THAN  OR  EQUAL  TO  ISEM.  LIST 
MUST  BE  ENDED  PITH  AN  •. 

SURPOilTlNF  INPUT 

COMMON  /COATA/  NRR  .NJ.NMT*  NAME (2.300> • A ( 100* 1001 .HI ISOOt . K J(I 01 I 
•  .  JNAMEI101».NSIEE(101>*JSUBLI100*IBn 

COMNON/IOD/MEAOEP(P»*NOTCOM.NBTVP1.NPTVP2*NFU2*NPRINT.JTIE 
C0MM0N/HELP/NV*INO(a00>*ICATf 300) .NAMIEt.JCHECK 13001 .ITT* IMAX.JMAX 
COMMON/  /MATR.THLO.SFMdOOI.ESPIlOO.lon 
COMMON  /HORX/X ( 100* 1001. SUMAI SOO) * JPANK ( loot *JPREF( 1001 
DIMENSION  SUMBOOOI 

RFAL  FVALUEdttOI 

LOOtCAL  ERROR*  JERR*  EDF 

DATA  ERROR/. FALSE./  *  jERR/. FALSE./ 

DATA  NOT/M  >«/«  NFQ/"  a»/ 

CLEAR  OATA 
JMAXalOO 
IMAXb300 
ERROR  a  .FALSE. 

N  a  0 

NOTCOM  a  0 
DO  10  Jal.lMAX 
HKJial. 

INOIJIaO 

10  ICATUIaO 

DO  II  Jal.JMAX 

MJIJIal. 

DO  II  lal.JMAX 

11  ACI.JiaO.O 
NJ  a  0 

00  15  K  a  1*101 
JNAMEIKI  a  IM 
NS17E(K)  a  0 
DO  IS  Jal.IOO 
ESRIJ.KtaO. 

IS  JSURLUtKI  a  0 

REAO  CONTROLS 

READ  IS.R12I  NpT.NC0MR*NRTYP1*NRTTP2*NATR*TMLD*NPR1NT 
HRITE  (6.91AI  NHT*NPTVP1*NPTV»?*MATR*THLD*NRRINT 
Rl?  F0RNAT(SI5*FS*2*I5» 

91 A  FORMAT!/"  NHT""* IP.SX.mnRTYRIb"* I2.3X ."NPTYRRb", I2*3X*»NATRa«*I2«S 
•X*"TML0a»*FA.2*3X*"NPRINTa»l?/) 

IF!  NCDMP  .NE.  0  1  HEAOERIRI  a  »  COMPLETED" 

IF(  NCOMP  .NE.  0  I  MRITE{A*91S) 
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TABLE  C-1.  (CCMiTUIUBD) 


^imRAUTlMr 


inn 


IM 


IQS 


?as 


3»»» 


?15 


??« 


INPUT  T4/T4  OPT«l 


*I»  POPM«T(/"  COXPLPTf  «LL  «UP-LI<TS"/) 

IF(  NPT  .se.  0  .OR.  NUT  .LC.  *  I  00  TO  TR 
WRITf(6t91*t  UWT 

91R  rORNRT(»*"tT55»"«*  C«ROR  IS  ILLCOOL  UflOHT  TTPCRI 

NUT  a  A 
TO  CANT  I  Nile 
ITTaO 
JMJaO 
INJaO 
NPUZaO 

OVBOSS  Fl*?ZT  IF  ueiSMTCO 
IF  INNT.NC.O)  NFUZal 

FNO  OF  MFIOMTS 


400  CONTINUE 

RE4A  «S«900)  JCLCtNECC 
IF  IJELE.CO.O.ANO.JNJ.AT.INJ)  NOTCONal 
IF  IJELF.EO.OI  NSRalTT 
IF  UELE.EO.OI  SO  TO  777 
«00  F0RN4T  tlStSltAiai 
URITC  (0.9011  NCLE 
901  FOPNOT  (/«  INPUT  READ  IN  "tAlO/l 
UPlTE(0«90Pt 

907  FORNAT  (/"  INOEA  EtENENT  NAHER*14X*auT"*10Xt''CAT*/I 
READ  ELEMENT  1N0E>*  MANES*  UEISMTS*  CATEOORICS 
17  READ  (S*904l  •(•NAM*NNI*MAT 
904  FORMAT  (1X«I4«9k«7A10*S«*FS.0»S«*IS> 

URITC  (0*9041  <*NAM*NHI*RAT 

IF(  NAPdl  .CO.  NCNO«  .09.  M  .EO.  *99  »  SO  TO  SO 
IF  (jCLC.CO.Zt  ITTaN 
IF  (JCLE.EO.lt  RrITT.K 
IF  (R.LC.lNAXI  SO  TO  70 
URITC  (0*900)  INAK 

900  F0RN4T("4«»TS5»«R*  ERROR  ••  IMOCX  LARSCR  TMAN"I4» 

FRSOR  a  .TRUE. 

SO  TO  40 

20  |F(  X  .ST.  0  )  SO  TO  2S 
URITC (0*007) 

907  F0RNATI"4"*T9S."aa  ERROR  ••  INDEX  LESS  7MAN  !•) 

ERROR  a  .TRJE. 

SO  TO  40 

PS  IF(  INO(RI  .Ca.  01  80  TO  90 

NRITC(0(900I  K*  NAME(1*X)*  NANC(2*K) 

90R  FORMAT ("•"«TSS*Naa  ERROR  ••  IN0CX”I4  *«  HAS  ALREADY  SEEN  DEFINED** 
•  •  AS  *(2A10I 
ERROR  a  .TRUE. 

SO  TO  40 

30  INO(K|  a  X 
ICAT(K)aXAT 
NAMCdtX)  a  NAMdl 
MAHC(2*R)  a  NAM(E1 
NaMAXO(N«XI 

IF(  UHl  .NC.  0.  I  UllXt  a  UHI 
IF(  NUT  ,E«.  7  )  UI(K)  a  UNI 
40  SO  TO  IT 
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TABLE  C-1.  (COMTINUBD) 


^imwOUTIMF  INPUT  74/74  RPTbI 


FTN  4.4*4S« 


p:i« 


PIS 


P4n 


P4« 


?4e 


9W 


P«« 


PM 


?7» 


PT5 


?40 


?M 


NO  NPP  BN 

IP  (JELC.fO.I)  NPRaNM^ITT 

Qo  4a  jBi.Naa 

irt  iMOcji  .ar.  a  i  so  ro  «a 

MRITE(««9ia)  j 

aia  FORMAT  ••eapORBB  hhat  is  elencnt  name  for  rN0EVB«i4i 
ERROR  B  .true* 

40  CONTINUE 
C 

i?a  continue 

c 

MRITE(6t«41l  HEADER 
•41  FOPM4r(«lM.M)0/) 

C  READ  SUB->Lt$T  RANKS  AND  JUDAES 

SOa  CONTINUE 

READ  (S*9aat  JUDAE*  JCONV*  HHJ*  ISEM 
930  FORMAT  (AIO*rS*FS.a*tSI 

IF  (JUOOE.Ea.RENOMI  SO  TO  400 
IF  UC0NV.E3.999I  AO  TO  440 
WRITE  14*9321  JUOAF*JCONV*WHJ*ISEN 

932  FORMAT  (//••0*‘*410*4X«».1CONV  bb,IS*4X*hJu06E  WEIAHT  BB*F4.1*4Xf  "USE 
•VALUE  LIMIT  •••*IS» 

TF(  MHJ  .EO.  0.  .AND*  NMT  *NE*  7  )  HHJ  b  1.0 
IF  (MATR.NE.O)  NFIIPbI 
C 

90  S02  JbI.NOR 
JRANK(J)  B  0 
SOP  JCHECKIJI  b  0 
JERR  B  .FALSE* 

C 

PRINT  1«NBR«(NJ»1I 

I  FORMAT  |/IX«wTOTAL  NAR  ALT  bB, IS.SX."NR  THIS  JII08E  bw.ISI 
IF  IJC0NV.GT.9)  90  TO  ROO 

IF  (jconv.eo.t.or.jconv.eo.ri  so  to  too 

CALL  pram  (FVALUEfNVfNSR) 

C  READ  FREE  FORMAT  DATA.  END  SUB-LIST  HITH  • 

C  CONVERT  TO  INTBSER 

IF  (JC0NV.E9.S.0R.JC0NV.E0.4)  90  TO  400 
S4S  CONTINUE 

00  SIO  JbI.NV 

-f  B  fvalueiji 

L  B  lABSiKI 

IF(  L  *0T.  0  .AND.  L  *LE.  NOR  >  SO  TO  SOS 
HRITE 14*9341  J*  K 

934  FORMAT!"  ••  ERROR  ••  ENTRY  NUM9ER»"IA»"  MAS  ILL1B*L  PROJECT/" 

•  "PRODUCT  OF"* IS  » 

error  b  .true* 

JEPR  B  .true* 

SOS  IF(  JCHECKILt  *E0.  01  SO  TO  SOS 
HRITE(4*433I  K 

933  FORMAT!"  ••  ERROR  ••"*1S*"  ALREADY  RANKED") 

ERROR  B  .TRUE* 

JERR  B  .true* 

AO  TO  sie 
C 

SOP  JRANK!J)  B  L 
JPPEFIJ)  B  NOT 


TAKJE  C-1.  (CONTIMUED) 


^UNVOUTIME  IVUT  74/T4  0»r«I 


rTN  4«4«49« 


»9fl 


?«4 


944 


944 


914 


915 


940 


3?5 


990 


995 


944 


JCNECKtLl  ■  L 

in  4  .LT.  4  I  JPOEEIJI  m  9E3 
510  CONTlMIIE 

JPffEEdi  ■  » 

C 

4RtTF(4««47l  (  JPOFEIJt*  J44N4IJI*  JbI.MV  I 

442  PORM4TI/  /(2«t*?*I91//n 

C 

C  CMEC<  fob  COMPUETF  SM-LIST 

FLOS  «  0 

-40  51?  J  ■  l»55B 

51?  IF(  JCHECKIJ)  .EO.  0  I  FLOS  ■  1 
IF  (FL40.NE.4I  JNJbJNJ.1 
IF(  FL4S  .NE.  0  t  BBTTE(4.9441 
940  FOBMOT  </"  SadLIST  IS  I9C0»*»LETE"» 

C 

1F(  JEPB  I  60  TO  544 
C  STORE  SJ9-LIST 

NJ  •  NJ  •  1 
JN4ME(NJ)  ■  JU06E 
VSIZEINJl  •  9V 
UJ(NJ)  ■  tfHJ 
no  515  J  ■  1.9V 
515  JS'JOLCJ.NJI  ■  FV4LUE(Jt 
C  JCONVa?  S(I3LIST  mot  COmplETFO 
IF  (JC0NV.E9.?>  MOTCOMal 
IF  (JCONV.EO.EI  60  TO  540 

IF  (JCONV.E9.3.4M4.9«TB.9E.4.99.JC0MV.CO.4.«MO.M«TR.Me.4l 
•WRITE  l«.920i 

924  F0094T(/"  SELE  EV4Ltl4TI49  R90WI5ITS  CnMPLETIOM»/t 
IF  (94TR.NE.0i  00  TO  534 
IF  <JC0NV.E9.3.OR.JC0NV.E0.4l  SO  TO  516 
C 

C  complete  SUB'LIST 

IF(  MCOMP  .eo.  0  i  00  TO  544 
514  CONTINUE 

IF(  NV  .SC.  NOR  i  00  TO  544 
C 

IF  (JC0NV.E0.3.0B.JC0NV.e0.4l  IMJalMJ*! 

IF  (JC0NV.ME.4I  60  TO  519 
C  JC0NVa4 

C  MOVE  OVER  FOR  LEFT  INSERT 

00  SIR  Jal.MV 
WlaNBR>J*l 
l(?aNV-J*l 

SIR  JSlt5L(Kl.NJ)aJSURL(K?.NJ) 

Mao 

DO  514  Jal.MRR 
00  519  L«RU9BR 

IF  (J.E0.I4RS(JSURL(LtNJil)40  TO  514 

513  CONTINUE 

MaN.l 

JSUBL(N«NJIaO 

514  CONTINUE 

JSUBL ( 1 .NJt  aI4NS( JsgSL ( 1 .NJI I 

NVVaNBR 

00  TO  532 
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TAUiE  C-1.  (CONTINUED) 


SUMRailTME 


17« 


7T5 


1^0 


1M 


INPUT  T«/7p  0PT«I 


PTN 


419  CONTINUE 
JCONtfaS 
QI6HT  INSERT 
NVV  >  NV 
DO  SaO  Kal.NSR 
00  S?n  Jal.NV 

in  <  .EO.  lABSt  JSUALIJ*NJI  >1  ftO  TO  SSn 
S?n  CONTINUE 

NVV  a  NVV  «  1 
JSUNLINVVtNJI  a  .  K 
s:in  CONTINUE 

JSURL(NV«1*NJ)  a  I«nS(  JSURL(NV*l«NJt  I 
53?  CONTINUE 

NSIZEINJI  a  NVV 
NV  a  NVV 

5*0  CONTINUE 

SELE  EVALUATION 
53*  CONTINUE 

IF  (MATR.EO.Ol  en  TO  5?5 
CALL  PRAM  (SEN*NVt-NRR» 

00  555  lal.NV 

IF  (SEMdl  .AT.lSEMt  60  TO  555 
555  CONTINUE 
90  TO  557 
555  VRITE  (6.9?II 

9?1  rORNAT  <»  •••ERROR***  SELF  EVALUATION  LEVEL  6REATER  TNAN  100  PERCE 
•NT"> 

90  TO  5?5 
557  CONTINUE 

00  5*«  1*1 tNV 

5*5  SENinaSENdt/FLOATdSENI 
00  5*7  Ial«NV 
00  5*7  JaI«NV 

5*7  IF  (J.E0.!*5S<JSURL(t«NJin  eSR(I*NJ)a5EM(J) 

WRITE  <*««5?|  (ESRdtNJIflaltNV) 

R5?  FORMAT  •/  /•t5(F5*?«3Xn 

THRESH  hold  matrix  REDUCTION  SELF  EVALUATION 
IF  (MATR.EO.il  60  TO  525 
00  5*6  Ial«N5R 

IF  (ESRdtNJi.LT.THLOl  90  TO  5*R 
60  TO  5*5 

5*9  IF  (JSUIIL(I«NJI.6T.O.*NO.JSU9L(I*1»NJ).LT.OI  JSUBLd*ltNJ)aI*9S( JS 
•UBL(I*ltNjn 
JSUDLdtNJIaO 
5*5  CONTINUE 
MaO 

00  553  lal.NSR 

553  IF  (JSURLdtNJI.NE.OI  MaM*l 
55*  Kao 

00  550  lat.NRR 

IF  (JSUBLdtNJI.EO.OI  60  TO  551 
90  TO  550 

551  IF  d.OT.MI  30  TO  550 
no  55?  Jal.NRR 
sal 

55?  JSU9LIJ«NJ|aJS09L(J*l*NJI 


TABLE  C-1.  (COHTIWJED) 


funtnUTI'tC  INPUT  74/T4  OPT»I 


PTN  4.6*439 


40A 


4n5 


410 


415 


4?0 


495 


430 


435 


440 


445 


450 


45S 


550  CONTINUF 

IF  (K.eO.ll  50  TO  554 
IF  (N.NE.NnOI  W9H 
MSIZE(NJ|aN9 
IF  (M.NE.NROI  NCOMPal 
595  CONTINUE 

C  OFOUIPEMENTS  TO  PBOJECTS  TR4N5L4TI0N 
IF  (JELE.EO.I)  00  TO  1050 
60  TO  500 
1050  C4LL  REQUIRE 
WRITE  <0.4511 

951  FORMAT  ( //IX.oREOUIRENFNTS  TO  RROJECTS  TRANSLATION"! 
00  1000  I"I.NV 
1000  JRANKdIaO 

no  1010  Jal.Htf 

KaJSUBL(J*NJI 

LalARSlKI 

JRANTCJIaL 

JPRFP<J)aN6T 

IF  IK.LT.01  JBRFFIJlatlEO 
1010  CONTINUE 

JPRFFIlla"  « 

WRITE  (0«4501  IJPRFKUltJRANKIJltJal.NVl 
450  FORMAT  </  /<2A|A2tI31//lt 

C  00  READ  MORE  OATA 

60  TO  500 
000  CONTINUE 
NVVaNV>l 
no  601  lal.MV 
601  SUMAlIlaFLOATIIl 
C  JC0NWa5 

C  CONVERSION  OF  OATA  T<1  DECENDIN6  OROFR 

IF  <JC0NV.E9.5>  00  TO  625 

no  605  lal.NVV 
Mal*l 

00  605  JaM.MV 

IF  <FVALUE<J).LT.FVALUE<I*>  so  to  605 
HOLOaFVALUEIJl 
SHOLOaSUMAfJI 
FVALOFlJlaFVALOEIIl 
SMNAIJIaSUMAdl 
FVALUE<IlaN3L0 
SUMA  < 1 1 bSHOlO 
605  CONTINUE 
MaO 

00  603  iBl.NV 

IF  IFVALUEdl.EO.O.l  SUMAIIIaO. 

IF  (SUMA dl.NE. 0.1  MbM*I 
603  CONTINUE 

MMaM»l 

00  602  lal.MM 

60?  IF  (FVALUEdt*CQ.FVALUEd*lll  SUMAd*lla-5UMAd4l) 
SO  TO  650 
C  JCONVaO 

C  CONVERSION  OF  OATA  TO  ASCENOINS  ORDER 

625  DO  610  lal.NVV 


TABLE  C-1.  (CONTINUED) 


W^POUTINir  I*t»UT  T4/74  OPTsl 


FTN  4.6*439 


no  610  JbM.MV 

IF  IFV«LUe(J».aT.FV«LUCII)l  93  TO  610 

MOLO>FV«UiE (Jl 

SHQLOaSUM*< Jl 

FV«LUC(Jt«FVALUE(ll 

SUM«(J)aSUM4(I> 

FV«LUE(I)aH3L0 

SUMOlItaSHOLO 

CONTINUE 

•<■0 

no  612  iBl.SV 

IF  IFV«LUE(II.Efl,O.I  SUM4|I)aO. 

IF  ISUHAdt.ME.O.t  9aM*l 
CONTINUE 

NMaM-1 

no  611  lal.NM 

IF  (FVALUEdI.EQ.FVALUEII*!))  $UNAd*lta.SiiNAd*n 
IF  (M.NE.NV)  NVaM 
DO  6S1  Jal.MV 
FVALUEI  J>>9'J<*A|J) 

60  TO  S4S 
CONTINUE 

CALL  PRAM  (C.NV.ll 
call  PRAMiS'JMa.NWfNOR) 
call  pram  (SUMA.NV.-NRR) 

IF  (JCONV.NE.Tt  60  TO  TOO 

JCONVa?  insert  AFTER  KEY  RE9UIRFMFNT 

no  TOl  lal.NV 

IF  (SUMAdt.Ca.  Cl  Kal 

00  70S  lal.NYV 

no  70S  Jal.K 

IF(SUMA(JI.C3.SUNadl|  SUMBdIaO. 

00  702  lal.K 

FVALUEdlaSUMAdl 

NRI|aNVV*K 

00  703  lal.NVY 

FVALUC<I*KlaSUMedl 

NVTaNV*NVV 

KlaK*l 

00  706  lal.NBB 
no  706  JaKI.NV 

IF  (FVALUFdl.EO.SUMA(J)l  SUMAfJIaO. 
no  704  laKl.NY 
F VALUE ( I *NVVI aSUMA I I I 
00  707  lal.NVT 

IF  IFVALUECII.EO.O.I  60  TO  TIA 

60  TO  707 

no  711  Jal.NVT 

FVALUE(JlaFVALUEIJ*ll 

CONTINUE 

MaO 

00  715  lal.NVT 
IF  (FVALUEd).NE.O.)  MaM*! 

NVaN 

60  TO  S4S 
CONTINUE 

JCONVaS  INSERT  BFFORE  KEY  REOUIRCNCNT 


TABLE  C-1.  (COllTIlJOED) 


^ufiRnuTiNr 

s»o 

s>^ 

S10 

S3S 

s*n 

^«0 

MO 


INPUT  7*/T4  OPT-1 


PTN 


no  7S1  laltNV 
7S1  ir  isuMAin.ro.  o  k«i 
K1»N-I 

no  7S5  IbUnvv 
no  7SS  JaNKI 

7N9  IF  ISUM<(JI.E3.SUMaCII)  SUMN(|)alt. 

no  752  lal.NI 
75?  rVALUEdlaSJNAdl 
NBRaNVV*Fl 
00  753  lal.NVV 

753  FV4LUEd*5naSUMndl 
NVTbNV*NVV 

00  75*  Tal.Nan 
no  756  JaKtNV 

756  IF  <FVALUFd»*EO.SUNA<J)»  SUN4(J)bO. 
no  754  laK.NV 

754  FVALUf  d«NVV>BSUMAd) 

00  757  IbUnvT 

IF  (FVALUEdI.eO.A.t  fiO  TO  7*6 
60  TO  757 

7AA  00  761  JbI.NVT 

TAl  FVALUF<Jl«FVAt.UF(J«l) 

757  CONTINUf 
MbO 

00  765  IbI.NVT 

765  IF  (FVALUCdl.NE.O.I  MbN«1 
NVbN 

60  TO  545 
BOO  CONTINUE 

CALL  PRAM  (PVALUE*NV*>N66) 

NRI7E  (6t506l  (FVALUEdl  *  Ia|  (NV) 

666  FORMAT  l/lXtMCATEOORT  RANKM«/5Xtl5IF6,0»2X)/> 

IF  (JCONV. £9.10160  70  6?5 
IF  <JC0NV.C9.11I  00  TO  550 
IF  (JCONV.E9.t2)  60  TO  575 

C  JC0NVB9 

c  banked  reouirfments  in  ranked  CATEBORIES 

no  620  Ib2,NV 

620  IF  (FVALUEIII.LE.R.I  60  TO  521 
mbI 

00  501  lal.NV 
00  602  JbI.nRR 

IF  IFVALUCdl.NE.ICATfJl)  60  TO  602 

suPA  (M)  bfloat  (  ino  ( jn 

IF  (M.OT.NBRI  60  TO  66? 

MBfc.l 

602  CONTINUE 
601  CONTINUE 

00  803  IbI.NRR 
663  FVALUEdlaSUMAdI 
NVbNBR 
60  TO  545 

621  MRITE  (6.6231 

623  FORMAT  (/••  CATEOORtes  NOT  RANKE0.RR0N6  JCONV*/) 
60  TO  540 
525  CONTINUE 
C  JCONVslO 


.6*43R 
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u  u 


TABLE  C-1.  (CONTINUED) 


<<E  INPUT  74/r4  OPTbI 


PTN  4.t*43« 


C  Q4NKC0  RFOUIPfNeMrS  IN  UNRRNKfD  CATFeORICS 
DO  R40  Ib9<>IV 

R4fl  IF  (Fv«Lue(n.dT.«.i  eo  to  049 

IF  (JCLF.FO.n  60  TO  R41 
DO  6?6  iBl.NV 
SIIM6(I|bO. 

00  S?6  JB1.N9R 

IF  (I.E0.IC4T(JII  SUM6II}bSUMR(I>»1« 

R?«  CONTINUE 
00  TO  643 
041  IFbITT*! 

ILbNRR*ITT 
00  042  IbI.NV 
SUMBdIaO. 

00  B42  JalFtIL 

IF  (I.E0.IC4T(J>1  SUNBIIIbSUNBIII*!. 

042  CONTINUE 
043  CONTINUE 
NbI 

00  027  IbI.NOR 
SUNbO. 

00  027  jBl.NV 
SUM4 ( M 1 BFL04T ( I 1 *SUM 
IF  (SUM4IN) .OT.HBR)  60  TO  627 
IF  (M.EO.I)  60  TO  029 
NNbN-1 

00  020  Kb1«NN 

IF  (4BS(SUM4(N)I.E4.0BS{SUM4(iC)).*NO.OUM.FO.O.I  KKbI 
020  IF  (  OBO(SUNA(M)|.eO.  4BS I SUNA IK) ) 1  60  TO  B32 
IF  (SUM.NE.O.)  SUMA|N)b-6UMA|N) 

029  MbM«1 

IF  IKK.EO.l)  SUNAIM'DbAOSISUNAIN.!)) 

KKaO 

032  SUNaSUM*SUM4(JI 
027  CONTINUE 

00  03S  laltNBR 
B30  FVALUEIII  aSUNA(I) 

NVbNBR 
60  TO  540 
040  WRITE  (6«046| 

046  FORWAT  I/"  CATE60RIES  NOT  UNRANKED*  WR0N6  JCONW"F) 

60  TO  540 
050  CONTINUE 

JCONVall 

UNRANKEO  requirements  IN  RANKED  CATFBORIES 
00  B5S  Ia2*NV 

059  IF  (FVALUEIII. LE.O.)  60  TO  B56 
NbI 

no  051  iBl.NV 
00  051  Js1,N9R 

IF  (FVALUEIII. NE.ICATIJI)  60  TO  SSI 
SUMAIMIaFLOATIINDUIt 

IF  (FVALUF(II.EQ.ICATU>1))  SURAINIa-SUNAINI 
IF  (N.EO.NBRI  30  TO  091 
Man*  I 

051  CONTINUE 

00  B92  IbI.nBR 


TABLE  C-1.  (COMTIHUED) 


tURBOIITIMF 


*3« 


A«0 


INPUT  T*/74  0PT«1 


PtN  4.»*«3« 


MS?  PVMLUeniaSUMMlII 
NVaNMM 
60  TO  5*5 
MMM  UPITE 

60  TO  S4« 

MTM  eONTINIir 
.JCONVal? 

(IN64NKC0  PraUIMEMENTS  IN  UNMMNMEO  C4TE60PIE6 
00  M7M  Ia?«NV 

MTS  IF  IFV«LUF(I).3T.0.»  60  TO  66S 
00  677  Ia2.N9M 
677  INOlIla.INOd) 

00  N7M  Ial«NMM 
M7N  FV4LUF<I|aFL0*TllNnlI>> 

MVamOM 
60  TO  649 
649  MPITECM«M«*I 

644  F0RM4T  t/"  C4TC60MIE9  NOT  UNPANKEOt  HRONO  JCONV*/) 
60  TO  946 

777  CONTINUE 
RETURN 
END 


490 


TABLE  C-1.  (CONTINUED) 


SU4»0I)TINF  r»FO  T4/T*  9PT«1 


PTN  4.«*0« 


1 


10 


IS 


?n 


?s 


1# 


15 


*0 


*5 


SO 


ss 


C  COMPUTE  roroUFNCV  MOTPtX  POP  E4CM  SU8>LIST 

C 

SUBROUTINE  ppEO 
C 

COMMON  /COATS/  NPP  *NJ«NWT*  NAME (?*300i •«( 100* 1001 tHI OOO) tHJI 1011 
•  «  JNAMC<101)«NSI7Ef10n*JSURL(100tlOn 

COMMON/IOO/HEAOEP(S)«NOTCOM*N»TVPI*NPTrPE*NEUE<NPRINT«JTie 
COMMON  /H0P</X(100.100t«SUMA(30») tJPANKIlOOItJPREEdOOl 
COMMON/  /mATR*TMLO«SEMI1AA)*ESP(100«101» 

OATA  nOASH/**— — »/ 

DATA  NGT/»  >"/t  NEO/"  •*•/ 

C 

00  000  K  a  1,NJ 
NV  a  NSIZEISI 
JUOOE  a  JNAME(K> 

C 

00  502  Jal.NSP 
so?  JPANKIJI  a  0 
C 

no  510  JaltMtf 
I  a  JSURLCJtSl 
L  a  lARSMl 
JRAMKIJi  a  L 
JPPEEIJi  a  not 

lEI  I  .LT.  0  I  JPREFIJT  a  ME*) 

510  CONTINUE 

JPPEPdl  a  a  H 
C 

C  SU3-LIST  FPEOUENCT  MATRIX 

CALL  MATRIX  (N5R«NV*JU09E»JRAM)(*JPREr*X) 

C 

C  MEI6MT  FREOUFNCY  MATRIX 

IF  (MATR.E4.01  00  TO  500 
C 

C  self  EVALUATION  MEIGHTINO 
00  SIS  lal.NSR 
00  SIS  Jal.NSR 

IF  (NMT.EO.OI  X(I,J|a4.aX(I.JI 
515  Xd*J|aXd«Ji*ESRd*KI 

IF  (NPRINT.C0.2.0R.NPRINT.E4.4.0R.NPRINT.Fa.5)  00  TO  500 
MRITE  (0«900I  MEA0ER*JU00E«(JOREFIJ>.JRANK|J)tJal*N«RI 
ROO  F0PMAT(«1SU5-LIST  SELF  EVALUATION  FREOUENCT  MATRIXatlOXtRAlOt/Zl X. 
•A10<(T12t24(A2tI3i/l) 

IF  (MATR.E4.2>  HRITE(6t«10) 

910  format  (/"  REDUCED  MATRIX"/! 

MRITE  (6.901)  (U.Jal.NRR) 

901  FORMAT  (/"  "R0J"»(TT»20I6)) 

MRITE  (6.9021  (NDASH.Jal.NRR) 

902  FORMAT  (1X.T7.20A6) 

00  903  lal.NSR 

903  MRITE  (6.904)  I . (X( I. J) . Jal.NSR) 

904  FORMAT  dX.I3.(T6."I  ".20F6.?)) 

500  CONTINUE 

C 

C  FUNCTIONAL  MEIOHTINO 

CALL  MEIOHT  (NHT.NBR«mI.NJ(K).X.JNANE(X)) 

C 


170 


TABLE  C-1.  (COMTIMUED) 


«i|fl«OIITI»IC  74/T4  OPT«l  rTM  4.4*4M 


C  SUM  sur>list 

00  5?0  J  ■  ItNOO 
40  00  «?0  !  ■  ItMBR 

570  4(1*JI  ■  4(I«J1  « 
C 
C 

400  CONTIMtir 

4S  C 

RCTURN 

END 
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TABLE  C-1.  (CONTIHUED) 


nU««OUTI*tF  WATRII  74/74  3»7bI 


F7II  4.4*49« 


I 


9 


1« 


IS 


?0 


9S 


3« 


IS 


40 


49 


90 


99 


C  roeoUCNCV  SOFOII  FOR  eOCH  SUR-LIS7 

c 

SUROauFtNe  047RtX  IS9R.MV»JUD3etJ«0*tK*JFItFr,X> 

c 

c  !it7voe  ■  0  TO  coNsioeo  *«^ro  vorioiles  onlv 

C  ■  1  TO  CONSIDER  ALL  URR49REB  V4R1ABLFS  E0U4L. 

C  SRR  ■  TOTAL  HUMMER  OP  VARIABLES  IM  STUOV. 

C  JUDGE  ■  MAME  OF  SUR-LIST. 

C  JRAHK  ■  RROJECTS  RANKED  IN  ORDER  OF  PREFERENCE. 

C  JPREF  ■  preference  OF  RANKED  RROJECTS  t  >  •  ■  I. 

C  I  ■  FREOUENCV  MATRIX  FOR  SUM>LIST. 

C  NV  •  NUMBER  OF  RANKEO  VARIABLES  IN  SUM-LIST. 

C 

COMMOM/IOD/HEAOERI At .NOTCDM.NRTVPl .MPTVPE.MFUZ.NPRINT. JTIE 
DIMENSION  XdOO.lOOl  .  JRANKIIOOI  *  JPRCFdoai 
C 

DIMENSION  JRdOOI.  JPdOOl 
OATA  NOT/m  *"/.  NFO/*  ■«/ 
data  N0ASH/«— — m/ 

C  SELfCT  TYPE  OF  CALCULATION 
EOVAlrO.S 
AOVAL>1.0 

TF  (NPTVPl.EO.O)  60  TO  9 
EOVAlrO.O 
AOVALrO.O 
9  CONTINUE 
C 

NVV  ■  NV 
00  10  J-l.NVV 
JP(J|  ■  JRAN<(JI 
10  JPtJI  ■  JPREFtJI 
00  12  J  ■  l.NSR 
00  12  I  •  l.NBR 
1?  Xd«Ji  ■  <1.0 
NVMlaNVV-1 
DO  GO  Kal.NVNl 
I  a  JRIKI 
XV  a  I.O 

IF  (JP(K«ll.Ea.NEO)  XXaEOVAL 
N  a  K  «  I 
00  40  MaN.NVV 
J  a  JR(M| 

Xd.JI  a  XX 
X(J.llaAOVAL-XX 

IF(  JPIM.ll  .NE.  NEO  >  XX  a  1.0 
40  CONTINUE 
00  CONTINUE 
C 

IF  INPTVPl.EQ.ll  NFUZal 

IF  IMPRINT. OE.I.AND.NPRINT.LE. SI  60  TO  90 
00  14  laltNBR 
00  14  JaltNBR 

IF  (NPTVP1.EO.O.ANO.RII«J).CB..9I  JTIEbI 
IF  (NPTVPl.Ca.l.ANO.XII.JI.CO.O.I  JTlEal 
14  CONTINUE 

VRlTElGtOOZI  HEADER.  JUDGE.  I  JPtJI.  JRIJI.  Jal.NVV  I 
90?  FORMAT ("ISUB-LIST  FREOUENCT  MATRIX”. POX. BAI0//1R. AIR. IT12.2AIA2.IS 


TABLE  C>1.  (CONTIMUBD) 


^ummuTtNC 


60 


65 


TO 


VOroiK  T4/T4  oor«t 


OTH  4.6*45« 


•  )/») 

rfRITE(6««04)  I  J*  J«t ••<(««  ) 

404  POR<44T(/M  049J««(  TT.EOIO  )  t 
KRtTE(6«40Sl  (  ND4SH*  J«1*N44  » 

405  r045*T(lK«T7«E0«6t 

no  TO  IaltN44 

M«ITE(6t406|  1*  (  XllcJt*  Ja|«  400  ) 

406  rO»5tT  (lX«I3t(T6*«I  •••a(OEA.ll» 

TO  CONTINUE 

44  CONTINUE 
return 

ENO 
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TABLE  C-1.  (CONTINUED) 


SUnnWITIVF  V^IAHT  74/7*  0PT«1 


PTN  4.»«*S« 


I 


5 


15 


?« 


?5 


5A 


35 


40 


45 


50 


55 


C  «ei6HT  POEOUCnCV  M*T5tX 

c 

5U500U71MC  Pei9H7  {Nt<T>'«95*MI»HJ*X*JN«MC> 

C 

C  N«T  ■  HEISHTINS  TVPE 

C  N5R  ■  NilMBER  OP  VA414PLE5 

C  «I  a  ROX  PEIQHT 

C  «J  a  JU08E  HEIOHT 

C  X  a  SJ9-L1ST  PREOUCNCV  5*751 X 

C 

C0MN0N/I0D/5EA0ER  till  •NOTCOPtMRTVPl  •l«PTVP2*NFU7«NPI(INTt  JTIE 
niME5SI0N  OlltOOl'-.  XllOO.lOOl 

0*7*  ND«SH/a— — .a/ 

C 

TF<  N«7  .L7.  1  .OR.  *M7  .07.  *  I  RE7UR5 
NJJ  a  NJ 

C 

C  MUCr  5«7RfX  RV  *  BEFORE  MEI6H71NA 

00  90  fal.RRR 
no  90Jal,NBR 
90  X(ItJta4.*X(I.JI 
C 

AO  70  (  100«200«300*40**500t*BOt700*AOO  >  NW7 

C 

100  00  IPO  I  a  1.58R 
Mil  a  Midi 
00  120  J  a  l.NBR 

120  Xtl.JI  a  Mtl  •  XI1*J> 

80  70  900 
C 

200  00  220  J  a  l.RBR 

00  220  I  a  I.RBR 

220  XlItJl  a  «JJ  •  Wtitl  •  XdfJI 
AO  70  900 
C 

300  no  320  I  a  t.RBR 
all  a  WKI) 
no  320  J  a  I.RRR 
IF<  X(I.J)  .LE.  0.0  I  AO  70  320 
X(1«JI  a  XdtJI  ••  ait 
320  C0N7INUE 
AO  70  900 
C 

*00  00  *20  J  a  l.RBR 
no  *20  I  a  I.NBR 
IFI  Xd.JI  .UF.  0.0  I  00  70  420 

xdfJi  a  Xd.JI  ••  f  aim  •  ajj  > 

420  C0N7INUE 
00  70  900 


500  00  520  Jal.RRR 
00  520  I  a  l.NBR 
IF(  Xd.JI  .LE.  0.0  I  80  70  520 
Xd.JI  a  aid!  •  (  Xd.JI  ••  ajj  I 
520  CONTINUE 
BO  70  900 


C 


TABl.E  C-1.  (CONTINUED) 


fURBOUTINF 


AO 


T« 


1% 


AO 


AS 


90 


AEICKT  74/T4  0PT«1  ^TN  4.A«A39 


AOO  00  690  J  A  l*9Ail 
00  690  1  >  i«saR 
in  XtltJt  .CG*  O.A  )  60  TO  690 

XlltJ)  •  «JJ  •  (  X<t«J)  ••  Atin  t 
690  CONTINIIC 
00  TO  900 

TOO  00  T90  Jal«N96 
00  790  I  ■  t*NBR 
tr(  xdiji  .ea»  o.o  »  oo  to  7»o 

X(1«J}  ■  X(t«JI  *  Mim  * 

790  CONTINUE 
60  TO  400 

AOO  00  090  J  ■  1*NA6 
no  690  I  ■  t*HB6 
in  xdtjt  .le.  0  I  60  to  abb 

XdtJI  a  ALOSI  Nidi  a  OJJ  •  Xd*J>  ) 

690  CONTINUE 

A6INT  WET8HTE0  NAT6IX 
900  CONTINUE 

ir  (N96INT.E9«9.06.N96INT.E0.4.00.N96INT.E0.S>  BETUBN 
M6ITE  16*4091  HCA0E6*JN4Nr*IJ»jal*N66t 
409  P06N4TI///aiAEI6HTC0  SUA^LIST  POEOUENCV  NATBIXAttOXtBOlB// 
•9X*A10«//»  •60ja,(TT«90t6tl 
M6ITE(6«40SI  I  l»6SH*  Jat*NBO  I 

905  P06N4TdX*T7*90A6t 

00  990  laltNBO 

«RITE(6«906t  1*  I  XdtJl*  Jal.NBB  t 

906  P06HAT  dX*I9*(T6*«l  «*9096*l)l 
490  CONTINUE 

9ET06N 

ENO 
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! 

TABLE  C-1.  (CONTIWJED) 


^UnaOUTlxe  OROFP  T«/T4  OPT«l 


FTN 


1 


10 


IS 


70 


2S 


30 


3S 


*0 


*S 


SO 


ss 


c  RRT3T  rank  ORDER 

C 

subroutine  ORDER!  NAME  •  NBR  t  SUMA  I 
C0MN0N/RAN(t/LlSTC(ia4*3l*LISTI10Bt*LkBI3l 

CDMM0N/10D/HE*DER|RI»N6TC0M»N»TTRl»MPTrRE»MrUEtMRRINT»JTIE 

C 

DIMENSION  SJNA!  1  » 

DIMENSION  JRANK!|«0»*  JRREFIlOOi 
C  SORT  SUMA 

C 

DO  ?00  JbI«NSR 
JRREFIJI  «  »  >" 

200  JRANK(J|  ■  J 

JPREFdt  ■  H  M 
C 

DO  210  JaltNSR 
DO  210  KaJtNSR 

IF!  SUMAiJt  .BE.  SUMAiK)  I  BD  TO  210 
TEMP  a  SUMACJi 
JP  a  JRANKIJ) 

SUMAIJ)  a  SUNAIKI 
JRANKCJI  a  JRANK(K| 

SUNAIKI  a  TEMP 
JRANKIK)  a  JP 
210  CONTINUE 
C 

IF  (NPRINT.C0.5.ANO*NAME.NC."PREFM|  RETURN 
IF!  name  .ED,  "  ••  I  RETURN 
C  STORE  FOR  CONCORDANCE 

IF  (NAMF.EB.mFUZZYhi  LASOIpNANE 
IF  (JTIE.EO.t**NO.NAME.EO*M*OJM|  L*B(1|mN«ME 
IF  IJTIF.EO.O.ANO.NAME.CB.MBORDA")  LOBIDbNAME 
DO  21S  Jal.NBO 
LlST(JlaJRAN<<Jl 
21S  CONTINUE 

00  2IB  JaPtNBR 

210  IE  (SUMA(JI.SO«SUMA(J»l)>  LIST|J|b.|.ISTIJ> 

DO  217  Ial«NSR 

IF  UTIE.EO.I.ANO.NAME.EQ.mBORDON)  00  TO  2lT 
IF  IJTIE.EQ.O.«NO.NAME.ED.”BORDAN)  LISTCII.llaLlSTin 
IF  (JTIC.EQ.I**NO.NAME.CQ.”ADJ"l  I.ISTC(I*11«I.1STII) 

IF  <jtif.eq.o.*no.name.eo."adj»>.  bo  to  217 

IF  (N«ME.EQ.«PREFM|  L1$7CI|*2Ib|,ISTII) 

IF  <NAMC.C0.mFUZZYM)LISTCII*3IbLIST(I) 

217  CONTINUE 

IF  INPRINT. EO. 5)  RETURN 
DO  220  Ja2tNSR 

220  IF!  SUMAIJI  .EO.  SUHAIJM)  1  JPREFIJ)  a  a  a» 

IF  (NAME.EO.MAOjai  NAMEbMAOJ  BORDA* 

NRITE(6tR0BI  NAME.  I  JPREFIJ) «  JRANKCJI*  JaltNRR) 

ROB  rORMAT(//UA10*  (T12t?A(A2tI31 /F)  1 
HRITEIBfROni 
ROR  FORNATdXl 

IF  (NAME.Ce.HAOJ  ROROAN)  NANCaaAOJH 

C 

RETURN 

END 
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TABLE  C-1.  (CONTINUED) 


Sim*OUTIME 

1 

s 

10 

15 

90 

95 

30 

35 

«0 

«5 

50 

55 


•HCr  T*/74  0PT«1 


COMPUTE  PPCPCaCMCf 
nUBOOUTINE  PPCE 

COMMON  /COOT*/  NOP  •NJ.NPr*  N«Me(Et3AO|»A<100*100)«Pt(SOO>tMjnPl> 
•  .  JN*MEI101l*NSIZEn0l>tJ5UBLI10i*101) 

COMM0N/I0D/He00EP|Pl*NflTCOM*NPTrPl*NPrvPP«MP|l2«NPPlNT*JTIP 
COMMON  /WORK/K 1 1 00* 1 Oil •SUM* ( 3001 • JR«NK  1 1001 • JPREE ( lOlt 
DIMENSION  Z7ET«(57t«PP|P7>«NTET«|P> 

007*  nOOSM/m— — — «•/ 

HOT*  PP/1.00,0.000»*.095«0.375**.000,1.00e«0.703»0.0«««O.M** 
•O.llT.O. 000.1. 000*0. T73*0.50**n.3**»0.E0R*0.1»R*0. 051 •O.OBZtO.OOO* 
•1. 000(0. 9«4«0.S53«0*T3T*0.553*e. *90*0. ENT.0.19N*e.ltt*0.000»t*eS3* 
•0. 017. 0. 000(0. U02*0*0a0*l. 000.0. 949(0. PS9.0.T«O*0.OP9*t.SP0*0.399* 
•0. 299(0. 200. 0.153(0.094(0.003(0. 037(0. 093*0. 011 (O.OOOOtO.OOEO* 

•0. 0013(0. 0004(0. 0001 *0.000*1. 000(0. 9«T0(  O.9R0(O.94S(O.OB2(0(OO3( 
•0. 702(0. 01 1(0. 490. 0. 405(0.320(0. 240(0. lO3(O*13a(O(095(O.OOT(0.O4S( 
•0.030(0. 019. 0.012(0. 007. 0.0042(0. 0023(0. 0013(0. 0000(0. 0003* 

•0.0001 (0.0001 (0.0001 (0.9001 (0.000/ 

9«T«  7ZETA/0. 000(1. 000(0. 000(0. 500(1. 000(0. 000(0. 200(0. 000(0(000( 
•0.000. 1.000* 0.000. 0.125(4. 250(0. 375(0.500(0. 529(0. TS0(0(*T5(1. 000* 
•0.000.0. 072. 0.143.0. 214(0. 255.0. 357(0. 429.0. 500(0. »T2(0. 043(0. TiS( 
•0.707(0. 055(0. 929* 1.000(0. 000(0. 050(0. 100(0. 1S*(0.200(0. 250(0. 300( 
•0.350.0. 400(0. 450(O.SOO(0.590(0. 000(0. 050.0. 700(0.750(0.000(0.a90( 
•4. 900(0. 950(1. 000(0. 000..0. 033(0. 000(0. 100(0. 133(0. 100(0.200(0.239( 
•0.26«(0. 300(0. 333(0. 300(0.400(0. 433(0. 460. 0.500(0.533(0. 500(0.000( 
•0. 033(0. 605(0. 700(0. 733.0.700(0. 000(0. 033(0. 060(0. 900(0. 933(0.500( 
•1.000/ 

OAT*  NZET*/0(0(1(3(0(12(21(30.57/ 

SELECT  TYPE  OF  CALCULATION 

EOVALaO.S 

LTVALmO.O 

*ftV*L«1.0 

IE  (NPTYP2.Ea.0l  60  TO  90 
COV*L«0.0 
LTVALm'1.0 
ADVALmO.O 
90  CONTINUE 

DO  130  Jsl.NSP 

00  120  I«J(N5R 
XIJ  ■  Ad.JI 
IJI  ■  *(J(II 
Xd.JI  ■  O.O 

lEI  I  .EO.  J  i  60  TO  120 
IE(  XIJ  .NE.  XJI  1  60  TO  no 

IE  (XIJ.EQ.0.0  .AND.NPTYPI.EO.O)  SO  TO  120 
Xd.JI-EOVAL 
X(J.I)aEQV*L 
60  TO  120 
no  V  a  1.0 

IE  IXIJ.LT.XJII  VaLTVAL 
Xd(J)  a  V 
X(J(Ila«0VAL«V 
120  CONTINUE 
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TABLE  C-1.  (CONTINUED) 


PBFF  7*/T*  OPT«l 


FTM  4.A**3« 


«0 


70 


75 


00 


05 


40 


45 


100 


105 


no 


130  C0N71NUE 

c  SU5  *(n 

00  150  Ial.434 
SUMO  ■  0.0 
no  140  Jal.MSO 
140  SUMO  ■  SUMO  *  X(I«JI 
150  SUMAlIt  a  SUMO 
C 

IF  (N00iN7.e9.SI  00  70  170 
M017E(6t90?l  ME40EO.  (  J.  Jal.NOO  t 
900  F0OM47  (»1C0MPUTE0  PREFEOENCE  M4TOT*".80X.O*10//"  SUM". (T13.19t«i 

•» 

MPI7E(A«905>  (  N04SH*  Jal.NOO  ) 

405  rORM«7  (1X.(714.14«0»I 
no  160  lal.MOO 

160  M0I7EI6.904I  SUMAdl.  I.  (  XlltJ)*  Jal.NOO  ) 

404  FORMAT  (lX.F6.1«l4*(T13.at  a.ioFA.m 
170  CONTINUF 

c 

call  order (  "POEF  «  .  NOR  •  SUMA  t 
IF  (NPOtNT.EO.SI  RETURN 
IF  (NPTVPF.Ea.OI  60  TO  2E0 
C  TEMP  1  *  0  •  -I 

XNMl  a  nor  •  1 
no  ?10  Jal.NSR 

Ein  5UMA(J)  a  0.5  •  <  SUMAIJ)  •  XMMl  ) 

?20  CONTINUE 
C 

C  PROCEDURE  FOR  ZETA 

C 

C  XX  a  NOR4(NR4-li4(NIIR*NRR«l)/ie.O 
XX  a  N0R4(NS4«n4(NOR»2}/«.0 

T2  a  24.0  /  (  NORaNOR4NSR  •  MtR4(  4-34MOD(NOR.2)  »  ) 

C  find  NUM9ER  OF  FROCTIONAl  SUMS 

NF  a  0 

no  230  J  a  l.NSR 
SA  a  J«  •  SUMA(J> 

230  tF(  SA  .NE.  SUMA(J|  I  NF  a  NF  •  1 

IF(  M00(  NF.2  I  .NE.  0  I  MRITE(6**10)  NF 
910  FORMAT  (/"••••ERR0R4PNUM9ER  OF  FRACTIONAL  SUMS  •"»13/) 

WRITE  (6.9501  NF 

950  FORMAT  (IX. "NUMBER  OF  FRACTIONAL  SUMSaa.lS) 

LE  a  3 

IF(  NF  .EO.  0  »  LE  a  1 
OS  a  0 
NR  a  1 
NRC  a  0 
C 

no  310  L  a  I.LE 

IF(  L  .EO.  3  >  60  TO  305 

JSUM  a  0 

1  a  0 

NHa2 

90  300  J  a  l.NBR 
SA  a  J«  ■  SUMA(J) 

1F(  SA  .EO.  SUMA(J»  )  60  TO  500 

C  FRACTION 
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TABLE  C-1.  (CONTINUED) 


«mflni)Tl*iE 

ns 

I 

i?o 

I 

1?S 

no  I 

I 

ns 

uo 

t«s 

no 

iss 

no 

ns 

no 


POEF  7»/70  OPT»l 


I  ■  I  *  1 

IF  (I.EO.NH)  tlOatlPC 
IF  (I.EO.NHt  N4sNH«2 
JO  ■  JA  *  NO 
SOO  JSUN  a  JSUM  *  JAajA 
NPal 

IF  (L.FO.21  NOaO 
SAaJA 

SflO  JSUM  a  JSUM  *  JAa|JA-l| 

0  a  XX  -  JSUN/7.0 
IF  (O.LT.O.)  aPITE  ntOOSI 
000  FORMAT  17130. 

IF  (O.LT.O. I  OaO. 

305  7ETA  a  1.0  •  0  •  Z7 

TEST  ZFT4 

IF  INBR.6T.9I  60  TO  ?5 
NSTARTaNZCTA(NaR) 

10  CONTINUE 

IF  (ZETA.EO.ZZETAINSTARTM  60  TO  30 

TF(ZETA.LT.ZZETA(NSTAPT»1>.AN'>.ZETA.6T.ZZETA(NSTARTI  )  60  TO  EO 

NSTAPTaNS7A0T*l 

60  TO  10 

interbqlftion 
ZO  CONTINUE 

Papp(NSTART«I)*(PP(NSTART)>PP(NSTART*n>P(7eTA*ZZETA(NSTART*in/ 
• (Z2ETA (NSTART 1 >ZZETA (NSTAPT*!) 1 
60  TO  40 
ao  PaPP(NSTARTt 
60  TO  40 
?$  CONTINUE 

6NU« (NBR4 (NSR'l I • (NRR.ZI 1 /( (nOO-4) ppE) 

CSa ( B . / ( NnR-4 . n • ( . ESP ( NSRP ( NB0>1 > • I NBR«E) /B . 1 aO* .S) *BNU 
PRINT  1.  CS.3NU 

1  FORMAT  </lX."CHI-SOUARE  aa,F10,3.SX."0F  aPFlO.3) 

CALL  MDCH  (CS.6NU.P.IER) 

Pal.-P 
40  CONTINUE 


IF( 

L 

.EQ. 

1 

1 

LAB 

■ 

"LOWER" 

IF( 

L 

.EO. 

Z 

» 

LAB 

• 

"UPPER" 

IF( 

L 

.EO. 

3 

) 

LAB 

m 

"AVERABE" 

IF( 

LE 

.EO. 

1 

1 

LAB 

m 

a  a 

IF  (O.LT.O. I  PRITE(6,03<I)  LAS  .O.LAB 
930  FORMAT  (/IX«"«ENOALL  0  BRACKET". AlO."  Da  ".FA.Pt  "THEBEFORE  ".AlO* 
•"MILL  BE  ZERO"l 

HPITE(A.90BI  LAB.  NBR.  0.  ZETA.  P 
OOP  FORMAT  (/lX.AlO."  Na  ". IS.SX. "KENDALL  D  •  ".FlO.ZtAX.RZCTA  a  a, 
•FlO.A.SX.nPROB  THAT  RANK  ORDER  NOT  CONSISTANT  a  a.FlO.R/l 
PTESTa.OS 
00  3?0  Jal.Z 
KNOTa"  " 

IF  (P.8E.PTEST»KN0Ta"N0T" 

WRITE  (O.OPOIKNOT.PTEST 
PTESTa.Ol 
3?0  CONTINUE 
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TABLE  C-1.  (CONTINUED) 


SliaQOIITINF  PaCF 


/?*  OPT«X 


PTN  4,»*439 


175 


140 


9?0  F0RM4T  IIX«hi)«NK  OBOER  “.RS."  CONSISTANT  4r«tF4.EtM  LCFEU"! 
MRsO 
RRC  ■  1 
OS  ■  OS  •  0 
n  >  05  /  9.0 
310  CONTINUE 


RETURN 

END 


TABLE  C-1.  (COMTIMUED) 


SURnOIITINF  r077V  T4/T4  OPT»l  FTHI 

I  c  Fuz;v  R«1l<  OMFR 

C 

■SUBROUTINE  FUZZY 
C 

B  COMMON  /cnAT4/  NRR  (NUtNUT*  N4NF  •«  ( in««  100)  t  RinOOi  vNJ  ( 1  HI  I 

•  •  JNAMF(101>«NSIZE<|01>tJSUBL(IOR*10n 

COMMON/ ton/HeAOeR(B)«NOTCOM»NRTVR|*NRTVPZtNFUZ*NRRINT*JTie 
COMMON  /WORK/y (100.100) .SUMAiaOO) tJRONMdOO) .JPREFIIOO) 

OATA  MOASH/M— — — 

10  C 

C  OIVIOE  EACH  A(T.U)  BY  NUMBER  OF  JUOSEB 

YNJ  ■  NJ 
00  no  Jsl.MBR 
DO  no  Isl.'^BR 

>  no  A(I.J)  ■  A(t.U)  /  XMJ 

C 

HRITE(6<40Z)  header.  (J.  JmI.MBR  ) 

00?  FORMAT  (hINDRMALIZEO  FREOUENCY  MATO t X-FUZZY». lAX.OAlO// (TO.EOTO) ) 
WRITE (6.904)  (  NOASH.  JbI.NBO  ) 

I  004  FORMAT(T6.ZOA6) 

DO  IZO  Isl.MOO 

t?0  WRITE(6.906)  I.  (  Ad.J).  JmI.NRR  ) 

906  FORMAT  (1X.I3.  (T6.«t  M.ZOFA.D) 

C 

1  C 

TRACE  F  a  0.0 
TRACE  C  a  0.0 
no  ZOO  I  a  1.<49R 

00  ZBO  J  a  I.NBR 

»  TRACE  F  a  TRACE  F  ♦  Ad.J)  •  A<J.I) 

TRACE  C  a  TRACE  C  «  Ad.U)  a  Ad.J) 

200  CONTINUE 
C  COMPUTE  F(R) 

FR  a  Z.O  •  TRACE  F  /  (  NBR4NBR  •  NRR  ) 

1  C  COMPUTE  C(RI 

CR  a  2.0  •  TRACE  C  /  (  NBR*MBR  •  NRR  ) 

C 

WRITE(6.91Z(  FR 
WR1TE(6.914)  CR 

s  012  aORMAT(/«  FIR)  aap^.j) 

014  FORMAT!/"  C(R)  a«.FT.l) 

C 

no  310  J  a  l.NSR 
XX  a  0.0 

D  DO  TOO  I  a  I. NRR 

XX  a  AMAXK  XX  .  A(I.J)-A(J.I)  ) 

300  CONTINUE 

RUMA(J)  a  1.0  -  XX 
310  CONTINUE 

0  C 

WRITE(6.016)  (  J  .  Jal.NRR  ) 

WRITE(6.01B)  (  SUMA(J)  .  Jal.NBR  ) 

916  FORMAT!///  IX. "PROJECT". (TIP. ?OIB)  ) 

OIR  FORMAT (IX. "FUZZY  RANH"  . (TIA.POFS.Z)  ) 

B  C 

CALL  ORDER!  "FUZZY  »  .  NBR  .  SUMA  ) 


TABLE  C-1.  (CONTINUED) 


FII77V  7*/T* 


BCTURN 

FND 


OPTal 
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TABLE  C-1.  (CONTIIWED) 


SUqvnUTIXF  CANCOR  T4/T4  OPTsI 


I 


10 


IS 


90 


9S 


30 


3S 


40 


4S 


90 


95 


9UB0AIITINC  CONCOR 
C 

COMMON  /COATO/  NRR  tNJtNtfT*  NAME (?t 300 1 t 4 ( | 00* lOOl tril OOOl f tf J M •!> 
•  t  jN4MEiion»NsiZE(ien»J9UBt(iootiei) 

c 

C0MM0N/ID0/HE40ER(AI*N0TCnM,N»TVRl*NPTVP2tNru7iNRRINT.JTIE 

peal  KSURLdOOtlAl) 
rOUIVALENCE  (  XSUAL*JSUaL  ) 
r. 

DIMENSION  RdOll*  JTEMBIlOOt 
DIMENSION  CCM(20.T«2t 
DATA  NDASH/Mo—..**/ 

OATA  (CCH(J«3«ll«J>I*20>/0.*T.A.10.*25.>3l.«3A.tA3.*4B.l*94.««0.* 
•4«..7I.9>77.9* A3. P*89.Ai99.0« 102, • tor. 7*1 1 3.7*119.7/ 
D4TA(CCW(J*4*l)*J«1.20l/e.*I9.5*35.*A9.9.AP.A.75.7.AP.7*101.7* 

•114. A 

•.127.A.140.A.193.a.100.9.179.R.14B.R.EOS.9.21B.9.232..245.«29B./ 
DATA  (CCU(J.5.1>*J«l*20)/0..37..A4.4*AB.4t|l?.3*136.1.l99.9.1S3.7* 
•207. 4. 231. ?. 254.9. 27A. 6*302. 4. 3PA.1.349.P. 373. 9*397.3*421. .444. 8. 
•460.9/ 

DATA  (CCW(J.6.1>.J«1.20>/0..  94. *  103.9. 143.3* I A2.4.221 .4*260.2* 
•299.. 

•337. A. 376. 7. 415. 9*494. 2*493. *931. 7*970. 5*609. 3*040.1 *600. B.72S.0* 

•764.4/ 

OATA  (CCM(J*7*lt*J>l*20)/0.*  97. *157. 3*217. *270.2*339.2*394.2* 
•453.1* 

•512. *571. *629*0*604. 0*747. 3* 406. 1*064. 9*923*7*902.4* 1441. 2*1099. 9* 
•1158,7/ 

OATA  (CCW(J*3*2t*J>l*2O)/O.*7.9*19.*31.*41.*92.*09.4*OO.O*TS.9* 
•AS. 1*94. 3* 103. 5*112. 7*121. 9* 131. *140. 2* 149.4*150. 0*107. 0*177./ 
OATA  (CCN(J*4*21*jMl*2e)/0. *19. *42. *01. 4*00. 9*99.9*110. 4*137.4* 
•156. 4* 175.3* 194. 2*213. 1*232. *250. 9*209. 0*200.7*307. 0*320.4*349. 3* 
•364*2/ 

OATA  (CCM(J*5*2)*J«1*20  1/0. *39. *75. 0*109.3*142*0*170*1*209. 4* 

•242. 7, 

•275.9*309.1*342.3*479.5*400.7*441.9*475*2*500.4*541.0*574.0*000.* 

•641*2/ 

DATA  (CCH(J*6*2)*J>l*20l/0.*  64**122.0*176*2*229*4*202.4*339.4* 
•300*3 

•*441. 2*494.. 546.0*599.7*692.5*705.4*750. 2.011. *003.0*910.0*959.4* 

•1022.2/ 

OATA  (CCH(J*7*2l*Jsl*20;/0. *104. *109. 6*209. *343. 0*422.6*501.2* 
•979*9* 

•650*4*737**015. 5*094. *972. 9*1051. *1129.5*1200. *1200. 5*1304. 9* 
•1443.4*1921.9/ 

C 

C  NJ  ■  NDMOER  OF  JUDGES 

M  ■  NJ 

C  NOR  a  NUM4ER  OF  PROJECTS 

N  a  NOP 
NSUMT  a  0 

C  GENERATE  EVALUATION  TABLE 

DO  90  J  a  1*N  . 

C  SIZE  OF  SUO-LIST 

L  a  NSIZEIJI 

C  STORE  SUO-LIST 

DO  30  K-  a  l*L 


1S3 


TABLE  C-1.  (CONTINUED) 


^U^^OUTlkie  CONCO0  74/74  nPT»l  FTH  4.ft*49« 


10  JTFHP«K>  •  JSU3LIK*JI 
XX  ■  *1  «  L  *  1 

40  XX  ■  XX  /  ?.0 

C  COMPLfTE  Sll*-L15T  T40LE 

IF(  L  (EO.  N  >  00  TO  49 

I  ■  N  -  L 

ISUHT  ■  MSUMT  ♦  toi»l  -  I 
44  00  *0  K  ■  1.1 

40  XSUBLtK.Jt  ■  XX 
C 

C  ASSUME  MO  MATCMES 

4S  00  SO  K  ■  l.L 

70  J4V  •  IABS(  JTEMF(K)  ) 

XX  >  K 

SO  XSlIflKJAV.Ji  a  XX 
C  FIMO  MATCHES 

MEfi  a  0 

7S  00  00  K  a  2.L 

JV  a  JTEMP(<I 
IF(  JV  .67.  01  60  TO  60 

NE6  a  NEO  •  1 
J2  a  K 

SO  tF(  K  .ME.  L  >  00  TO  60 

40  IF(  ME6  .LF«  01  00  TO  00 

J1  a  J?  .  nEO 
XX  a  J1  «  J? 

XX  a  XX  /  2.0 

05  C  IMSFRT  matches 

00  70  I  a  J1.J2 
JAV  a  lABSI  JTEMPIII  > 

70  XSUBUIJAVtJI  a  XX 
C 

00  fa  NEO  *  1 

NSUMT  a  NSUMT  ♦  -  I 

NEO  a  0 
00  CONTINUE 
90  CONTINUE 

OS  C 

SUMT  a  NSUMT 
SUMT  a  SUMT  /  12.0 
C 

WRITE (6.902)  HEADER*  (  J*  Jal.N  ) 

100  90?  FORMAT  (nICOMCOROANCE  SUMMARY  BY  ELFHENTa  ?OXtRA10/a  UNWEISHTEO  SU 

•0L1STS"/(T12*?01A)) 

NRirE(4.904>  I  NOASH*  Kal.N  ) 

904  FORMATI"  JU09E  a*  (713*2044)  ) 

C 

lOS  00  100  Xal*N 

100  R(K)  a  0.0 
SUM9  a  0.0 
C 

00  n«  Jal*M 

110  IF  (JNAME(J).C3.aADja)  JNAMEl J)aaoDj  BORDAa 

WRITE (6*906)  JNAME(J)*  (XSUBUK*J)*  KaltN  ) 

904  FORMAT  (lXA10*(T13*al  »*20F6.1)) 

00  110  Kal*N 

R(KI  a  R(KI  *  XSURL<K*J) 
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TABLE  C-1.  (COHTINUED) 


sunonuTitr  concob  T4/74  optbi 


PTN  4.**43« 


11^ 


l?0 


1?^ 


130 


135 


140 


145 


150 


155 


160 


165 


ITO 


SUMO  •  SUMP  •  XSUnL(5*JI 
110  CONTINIJC 

c 

BP4B  ■  SIIMR  /  N 

WRITE  I6t506l  N04SH*  I  N04SM*  1 

40A  P0RM4T(T6«44«  (T13*M46I  I 

wRITEI6t«10l  (  R(KI*  K>1*W  1 
510  rORMOTC"  aiJl«*IT14»20  F6,l)  » 

C 

S  ■  0.0 
00  120  Kal.W 

120  5  a  S  *  <  R<<>  >  Rfl4R  | aa? 

WRITE (6.9121  RB4R.  $ 

912  F0RM4T  9€*W  aH.no.P.SX.asuM  OF  OEVIATIOWS  S0U6RE0  a«F10.»/l 
tfRITE(6.914)  SUMT 
914  FOR>44T(H  SUW  T  a",  F10,21 
C 

C  KCNOALS  COEFFICIENT  OF  CONCORDANCE 

Oa  ( waMa  (MaRait.mi  /I ;»)  .WassilMT/l  > 

Ca9999. 

IF  (O.NE.0.1  CaS/0 
C 

NOF  a  n  -  1 
C 

CHISO  a  M  a  mDF  a  c 
IF  (C.EO.9999.1  WRITe  (6*9171 

917  F0RN4T  (/1X.HKEN04LLS  COEFFICIENT  OF  CONCORDANCE  15  INDETERMINATE" 
a/1 
C 

WRITE(6«9161  C.  M.  N 

916  FORMAT!/  IX.hkENOALLS  COEFICIENT  OF  CONCORDANCE  "".FlO.S.SXi 
a  «M  a»I3,5X*»N  a"I3/) 

IE  (N.lF.7.AN0.M.tE.20l  50  TO  130 
C 

OFaNOF 

call  MOCH  <CNISQ*DF,R,iE9| 

Ral,-R 

WRITE  (6.9151  CHISO.NOF.R 

91M  FORMAT!/"  CHI-SOUARE  aM*F1fl.3,SX."DF  a«,l4*5X*"R  bN»F6.41 
RTESTa,05 
00  320  Jal*2 
KNOTa"  " 

IF  (P,GE.RTESTlKNOTa"NOT" 

WRITE  (6.9221  KNOT. PTEST 

92?  format  (IX. "RANK  ORDER  ".A3."  CONSISTANT  AT".F4,2."  LEVEL. "1 
PTESTa.Ol 
320  CONTINUE 
RETURN 
C 

130  CONTINUE 
TEST  a  .05 
RTESTaCCW(M,N.ll 
no  325  Jal.2 

KNOTa"  " 

IF  (S.LE.PTESTI  KN0Ta"N0T« 

WRITE  (6.9201  KNOT.  TEST.  PTEST 
TEST  a  .01 
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TABLE  C-1.  (CONTINUED) 


SURROUTtMr  C0NC9R  74/T4  OPTa]  PTN  4.ft*43« 


PTCSTaCCH(M*^««> 

3PS  CONTINUE 

9P(I  F0R94T  (IX«m94MK  OPOEP  »*4St  "  C0NS19T4NT  AT  "•F4.P*w  LEVEL*  CPTTT 
ITS  aCAL  S  ■  ».fT.?» 

9ETU9N 

END 


TABLE  C-1.  (CONTIimED) 


^UR^flUTlNE 

I 

R 

10 

15 

90 


COMPARE  T4/74  OPTbI 


PTM  4.M47* 


«U9P0UriNE  COMP ARP 

COMMON  /CDATA/  NRR  «NJ«NWT*  NAME IP*9A0)  t A llOO.IOOl  ,111  (30i»  .MJIIRU 
•  •  JNAME(lAl),NSI7En0n.JSUeL(10R«ltli 

COMMON/RANK /UlSTCil00«3)«l.tST|100l*LAl|(.R) 

OATA  LAR/RM  *4HPREF*RM  / 

NK«3 

MJa? 

IP  aA«l3» ,ME.“P027y"l  M<»P 
00  10  Kal*N< 

on  10  l«k*n< 

IF  (F.EO.LI  30  TO  10 

JNAMEdlaLARIKI 

JNAMEC2taLAR(L) 

NSIZEClIaNRR 
NSfZE  (PtaNRa 
00  3  tal.NRR 
JSURL(I«ll«LI3TCIt,iO 
JSURI.(I*ZtaLISTCIt»LI 
R  CONTINUE 
CALL  CONCOR 
10  CONTINUE 

LARdtaa  » 

LAR(3iaa  M 

RETURN 

ENO 
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TABLE  C-1.  (COHTIHUED) 


SU««OUTIMC 

1 

% 

1« 

\i 

so 

ss 


PFOUIOC  T4/T4  OPT«l 


rrnt  4.»443« 


SUBOOUT.INC  OEOUIRC 

COMMON/HELP/NVtlNO(3ftOi*lC«TI3»a)tB4M(?).JCHeCKOOei*ITTtlM4X,JB«X 
COMMON  /Cn*T4/  NMR  •NJtNMT*  N4ME(Et3001  *«  llOOt  lOOl  tMI  (3001 
•  •  JN4ME«l«n*NSIZE(lBn«JSUBL(100>10n 

OImENSION  TELEMI3ftB«B|,tSUH|300)*JPROJI300) 
i)*T»  IR/0/ 

IP  (IR.NE.OI  80  TO  SB 
RPHINO  9 
00  SO  Ial*30e 

REAO  (BilOO)  (IEt.EMII*j|«Jal,?) 

IP  (E0P(4t.9E*e.|  60  TO  40 

100  P0OM4T(2*10i 
IB>1 

60  CONTINUE 
IX>301 

40  NPLE«IP-1 

MBITE  («««BOI 

960  P0RM4T  <////T10«MTNOEX  LIST  OP  REOUIREMENTS  4N0  PROJECTS"! 

W9ITE  (O.lOli  (t*IELEM(I«ll*IELEM(I*PI*I>l*NrLE> 

101  F0RM4T  (/4(I4*2X*?4I0«M/«!) 

S6  IC«1 

lt«0 

00  OS  laltIMOX 
ISUMdIaO 
OS  JPROJ<I!aO 
no  11  Tal.NV 
Kal 4RS I JSUBL  < I •NJII *1TT 
IPaO 
M"1 

no  10  JaltNELS 

IP  (N«ME(ltSI.E«.IELCM(J*l)!  BO  TO  S 
60  TO  10 
S  riair#! 

JPR0J(II)alELEM(J«2) 
no  IS  MaltlTT 

IS  IP  (JPR0J(III.E0.N«ME  (l*M!iISUMIII!alNO(M> 

IP  (JSUBLtNHJI.LT.O.ONO.  N.EB.l)  ISUNlIIla-ISIISdl) 

IP  (N.6T.I!  ISUMdIla-ISUMdl) 

NaN*I 

IPaIP*l 

ICalC*! 

10  CONTINUE 

IP  dP.EQ.OMRITE  10*900!  N»MEI1*KI 
900  PDRMOT  (/1X«410*"  IS  NOT  IN  REOUIREMENT  INDEX"/! 

11  CONTINUE 

00  20  lal.IC 
DO  20  Jal.IC 
IP  d.EO.J!  80  TO  PO 

IP  dXRSdSUNIl!!.NE.IORSdSUNIJ!!!  80  TO  20 

IP  dSUM(J).8T.O.ONO.lsUMIJ*l!.LT.P!  ISUMI Ja] lalOBSlISUMI J*1 ! ! 
ISUMCJIaO 
20  CONTINUE 
"aO 

no  2S  lal.IC 

PS  IP  dSUNdl.NE.B!  MaN*l 
30  XaO 
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nuBnnuTiNF 


«<i 


61 


TO 


TABLE  C-1.  (CONCLUDED) 

oeOlltB^  74/74  OPT«l 


rTm 


00  3»  lBl«lC 

IF  (ISUMMI.EO.O)  BO  TO  31 
SO  TO  3S 

31  IF  (I. ST. Ml  30  TO  3S 
no  3?  J-TtlC 
»■! 

3F  I«llM(j|als*m(J*l> 

30  COMTINUF 

IF  IK.FO.l)  30  TO  30 
MVbM 

MSIZE(NJ>aM 
90  45  I-l.M 
45  J5U0L(I*MJIaISUM(II 
OETUSN 
END 


.0*439 
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